Zeekurity Zen – Part II: How to Send Zeek (Bro) Logs to Splunk
This is part of a larger series on building a Zeek (Bro) network sensor.
Zeek (formerly named Bro) is my favorite security monitoring platform, and I’ve used and promoted it throughout my career. It generates rich network metadata that’s incredibly valuable for incident response, forensics, and general troubleshooting.
Perhaps the main challenge with using Zeek is actually setting it up. While today there exists Corelight (an easy-to-use Zeek appliance with enterprise support), not everyone has the budget for something like this. This series will walkthrough Zeek setup, integration with Splunk, and various tips and tricks I’ve learned over the years.
In Part II, we’ll walkthrough several steps:
- Configure Zeek to output logs in JSON format for consumption by Splunk.
- Create an index in Splunk for Zeek data.
- Installing and configuring the Corelight For Splunk app to index and parse Zeek logs in Splunk.
- Create a splunk user to run the Splunk Universal Forwarder.
- Installing and configuring a Splunk Universal Forwarder to send Zeek logs to a Splunk instance.
Output Zeek logs to JSON
- Stop Zeek if it is currently running.
- Edit /opt/bro/share/bro/site/local.bro and add the following.
# Output to JSON @load policy/tuning/json-logs.bro
- Restart Zeek and view the logs in /opt/bro/logs/current to confirm they are now in JSON format.
broctl deploy cd /opt/bro/logs/current less conn.log
Create an index in Splunk for Zeek data
It’s best practice to create separate indexes for different types of Splunk data.
- Login to your Splunk instance.
- In the top right menu navigate to Settings -> Data -> Indexes.
- In the Indexes page, click on New Index.
- Type “zeek” for Index Name and click Save to create your new index.
Install and configure the Corelight For Splunk app
The Corelight For Splunk app is created by the Zeek team directly for use with Corelight (enterprise Zeek) and open-source Zeek sensors. We’ll use this app to help parse, index, and visualize Zeek logs.
Note that Splunk has also published their own Splunk Add-on for Zeek aka Bro app which helps to ingest Zeek logs but does not feature any sort of dashboards or reports.
- Download and install the Corelight for Splunk app onto your Splunk server. This can either be done within the Splunk server itself or by manually downloading and installing as you would all other Splunk apps.
- You can navigate to the app to verify that it is installed correctly. However, since we have not yet configured our sensor to send data, the dashboards will be blank.
- In the top right menu navigate to Settings -> Knowledge -> Event types.
- In the App dropdown menu, select Corelight For Splunk and click on corelight_idx.
- In the Search string field type index=zeek. This tells the Corelight for Splunk app to search for data in the “zeek” index we created earlier.
Create a splunk user to run the Splunk Universal Forwarder
Back in the Zeek sensor, create a splunk user and add it to the bro group.
sudo useradd splunk -g bro
Install and configure a Splunk Universal Forwarder
- Login to your Splunk account and download the latest Splunk Universal Forwarder. Once logged in, click “Download Now” for the Linux 64-bit .rpm installer. Note that Splunk also generates a convenient wget command you can use from the sensor itself once you accept the license agreement. As of this writing, the latest release is version 7.2.3. If the download URL referenced in the wget command below no longer works, download directly as noted above.
wget -O splunkforwarder-7.2.3-06d57c595b80-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.3&product=universalforwarder&filename=splunkforwarder-7.2.3-06d57c595b80-linux-2.6-x86_64.rpm&wget=true'
- Install the forwarder to /opt/splunkforwarder using the rpm command.
sudo rpm -i splunkforwarder-<…>-linux-2.6-x86_64.rpm
- Start the forwarder to accept the license agreement and create an administrative password.
cd /opt/splunkforwarder/bin ./splunk start --accept-license
- Stop the forwarder.
- Remove the default data processing limit. Edit /opt/splunkforwarder/etc/system/local/limits.conf and add the following lines.Note that given the volume of data that Zeek generates, the forwarder may never process all log data if the default limit is not removed.
[thruput] maxKBps = 0 # means unlimited
- Edit /opt/splunkforwarder/etc/system/local/inputs.conf to monitor desired Zeek logs. An example inputs.conf is below but may or may not include the logs you wish to ingest. Note that the index and sourcetype fields are leveraging the “zeek” naming scheme to match the “zeek” index we created in Splunk.
[default] host = sensor [monitor:///opt/bro/logs/current/conn.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_conn [monitor:///opt/bro/logs/current/dns.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_dns [monitor:///opt/bro/logs/current/software.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_software [monitor:///opt/bro/logs/current/smtp.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_smtp [monitor:///opt/bro/logs/current/ssl.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_ssl [monitor:///opt/bro/logs/current/ssh.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_ssh [monitor:///opt/bro/logs/current/x509.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_x509 [monitor:///opt/bro/logs/current/ftp.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_ftp [monitor:///opt/bro/logs/current/http.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_http [monitor:///opt/bro/logs/current/rdp.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_rdp [monitor:///opt/bro/logs/current/smb_files.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_smb_files [monitor:///opt/bro/logs/current/smb_mapping.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_smb_mapping [monitor:///opt/bro/logs/current/snmp.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_snmp [monitor:///opt/bro/logs/current/sip.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_sip [monitor:///opt/bro/logs/current/files.log] _TCP_ROUTING = * index = zeek sourcetype = corelight_files
- Edit /opt/splunkforwarder/etc/system/local/outputs.conf to send data to your Splunk server. In the sample file below, replace each instance of splunkserver:9997 with your own server name/IP and port number.
[tcpout] defaultGroup = default-autolb-group [tcpout:default-autolb-group] server = splunkserver:9997 [tcpout-server://splunkserver:9997]
- Make the splunk user and bro group the owner of the entire /opt/splunkforwarder directory.
sudo chown -R splunk:bro /opt/splunkforwarder/
- Start the forwarder as the splunk user and confirm it successfully ran. You can check /opt/splunkforwarder/var/log/splunk/splunkd.log for any issues.
cd /opt/splunkforwarder/bin ./splunk start
In Part III of this series, we will dig into Zeek data with sample Splunk queries.