How to Choose the Right Endpoint Protection Platform (EPP) / Endpoint Detection and Response (EDR) Solution

Introduction

Like most cybersecurity professionals, you’re looking for an EPP that protects against current and evolving threats, is easy to deploy and manage, and is ultimately invisible to end-users.  Today, there are dozens of these platforms available, and choosing the right one for your business is a daunting task.  With each vendor claiming their solution is next-gen-everything, is the highest rated, and is easiest to use, how do you select what’s best?  

Last year, I conducted a bake-off amongst three endpoint security solutions.  My goal was to replace an incumbent legacy system with a modern combined endpoint protection platform (EPP) and endpoint detection and response (EDR) solution.  I researched online for methodologies and guides to evaluate and compare solutions but didn’t find much.  The SANS Advisory Board mailing list had some good tips and together with my own curiosity and experience, I created the following walkthrough.  The entire experience was one of the most fun I’ve had and I hope my guide helps you in your own evaluation process.

Determine Requirements

First and foremost, determine your requirements.  Some questions to ask yourself:

  • What operating systems do I need to cover?
  • Am I worried about file-less malware and PowerShell-based attacks?
  • Is endpoint detection and response (EDR) capability important to me?
  • How will this new solution integrate and enhance my current solutions?
  • Do I need a solution that provides managed services such as monitoring or incident response?

Based on these questions, create a “capabilities checklist.”  This will be your specific criteria for a valuable experience.  A sample is below.

Capability Yes/No Comments
Single agent or multiple products?    
Search for IOCs: MD5/SHA256 hash, registry keys, filenames.    
View downloaded files, DNS cache, network connections.    
Live forensics: quarantine system, view / kill processes, download files for analysis, view active network connections    
Integration with existing solutions: Firewall, SIEM, etc.     
Whitelist files/directories.    
Run on-demand manual scans on files/directories.    
Process for handling false positives.    
Create groups of systems and apply specific policies per group/system.    
Disable an agent from the central console.    
Low resource usage.    
Preventing/detecting malicious PowerShell scripts, file-less attacks, suspicious command usage.    
Detect and prevent malware.    
Unique features of the platform.    

Research Solutions

There are plenty of guides, reviews, and reading materials available to get an idea of what solutions will meet your requirements, at least on paper.  Good starting points are:

  • MITRE ATT&CK Framework
  • Webinars / Conferences
  • Security publications
  • SANS Reading Room
  • Gartner / Forrester
  • Your security peers

Depending on your resources and timeline, try to narrow your list to 2 – 4 solutions you feel meet your minimum requirements. 

Vendor Communications

Reach out to the vendors you intend to evaluate.  Prepare a good list of questions and have them walk through your use cases.  Typically, these pre-sales calls will involve an account executive and a sales engineer that will introduce you to their solution, speak to their differentiators, and conduct demos.  You may even be able to weed some of the vendors out just in this initial stage.

Throughout the process:

  • Don’t reveal which solution you’re leaning towards.
  • Document your findings.
  • Ask lots of questions.
  • Be courteous.

Testing Methodology

Truth be told, no solution is perfect, but leveraging a standard testing methodology will enable you to objectively evaluate each solution in a fair and repeatable manner. You can use the following methodology to complete your “capabilities checklist” from the Determine Requirementsphase.

Preparation

  1. Setup test “victim” virtual machines, completely separated from your production networks, running each of the operating systems you’ll need to protect.  Virtual machines will allow you to quickly revert to a clean state. Cloud or container-based solutions work well for this.
  2. Setup an “attacker” machine using a Kali virtual machine.
  3. Download a collection of known malware samples (e.g. Petya). Lenny Zeltser has a great list of free malware sample sources.
  4. Set up a malicious PowerShell attackusing Unicorn (https://github.com/trustedsec/unicorn).

Active Testing

  • Run the known malware samples and observe which are detected and prevented.  Test this both on and off network to accurately evaluate any cloud or offline prevention capabilities.
  • Observe resource usage using the respective operating system’s process monitor.  Note resource usage during idle and malicious activity.
  • Run the Unicorn PowerShell attack using Kali.  Try this in both “full prevention” mode and “detect-only” mode to test prevention and EDR capabilities.
  • Run additional types of attacks using Kali: https://www.offensive-security.com/metasploit-unleashed/
  • Run suspicious commands like netcat or dump password hashes and note if the activity is prevented or detected.  Observe whether or not the solution brings these to your attention or if you have to dig to find what you’re looking for.
  • Test the ease of deployment and uninstallation.  Both will be equally important to any teams that will be supporting the management and maintenance of the platform.
  • Test “bypass”, “detect-only”, or “disable agent” modes in the event you’re asked to disable the protections for troubleshooting purposes.
  • Move a system between policies and observe how long it takes for changes to apply.
  • Test remote forensic capabilities.  View running processes, download files, kill processes, and view netstat information.
  • Observe how easy it is to search for IOCs:  hashes, filenames, IP addresses, hostnames.
  • Create whitelists / blacklistsfor specific files and test to see they are actually allowed or blocked.
  • Test network containment of an endpoint.  Is the system truly isolated and unable to connect to anything other than the EPP console?
  • Get a feel for the UI.  Are the features you care about the most easily accessible and intuitive to use? Is it slow or difficult to navigate?
  • Test any unique features of the platform.  Do they run as well as the vendor claimed?  Do they add value to your workflow?
  • Try to seriously break it.  Click on everything in the console.  Assuming you’re only running against test virtual machines, this shouldn’t break anything.  If you run into issues, reach out to the vendor’s support team and evaluate how responsive they are. Better to know this now than when you’re in a real emergency.

Final Evaluation

  • With your completed “capabilities checklist”, review your findings and observations.  If there are any follow up questions, get these answered by the vendors in writing.
  • It’s not an exact science, but the results from your checklist combined with your experience working with the pre-sales and support teams should give you a good idea of which solution is right for you.

Purchasing

  • Get quotes for all the solutions you’re evaluating. Even if you’ve decided on which solution you want to go with, use the other quotes as a way to drive down the price for the solution you want. Let each vendor know you’re looking at competing solutions and ask them to include any training, conference tickets, or additional incentives to create the most compelling offer. Don’t feel bad about this, it’s their job to sell to you.
  • Depending on your timeline, you may not have a choice for when to make the purchase, but typically you’ll want to go for the vendor’s year-end or quarter-endto get the best prices and incentives.  Obtain in writing what the expected renewal process and prices are.

Conclusion

Choosing the right endpoint protection platform solution is critical to protecting your business from today’s ever-evolving threats.  With an increasing number of EPP solutions to choose from, cutting through the marketing noise is a significant challenge.  By applying a standardized evaluation and testing methodology you can ultimately make an informed and objective decision on the right solution for your business.

Zeekurity Zen: Part II – How to Send Zeek (Bro) Logs to Splunk

This is part of a larger series on building a Zeek (Bro) network sensor.

Overview

Zeek (formerly named Bro) is my favorite security monitoring platform, and I’ve used and promoted it throughout my career.  It generates rich network metadata that’s incredibly valuable for incident response, forensics, and general troubleshooting.

Perhaps the main challenge with using Zeek is actually setting it up.  While today there exists Corelight (an easy-to-use Zeek appliance with enterprise support), not everyone has the budget for something like this.  This series will walkthrough Zeek setup, integration with Splunk, and various tips and tricks I’ve learned over the years.

In Part II, we’ll walkthrough several steps:

  1. Configure Zeek to output logs in JSON format for consumption by Splunk.
  2. Create an index in Splunk for Zeek data.
  3. Installing and configuring the Corelight For Splunk app to index and parse Zeek logs in Splunk.
  4. Create a splunk user to run the Splunk Universal Forwarder.
  5. Installing and configuring a Splunk Universal Forwarder to send Zeek logs to a Splunk instance.

Output Zeek logs to JSON

  1. Stop Zeek if it is currently running.
    broctl stop
  2. Edit /opt/bro/share/bro/site/local.bro and add the following.
    # Output to JSON
    @load policy/tuning/json-logs.bro
  3. Restart Zeek and view the logs in /opt/bro/logs/current to confirm they are now in JSON format.
    broctl deploy
    cd /opt/bro/logs/current
    less conn.log

Create an index in Splunk for Zeek data

It’s best practice to create separate indexes for different types of Splunk data.

  1. Login to your Splunk instance.
  2. In the top right menu navigate to Settings -> Data -> Indexes.
  3. In the Indexes page, click on New Index.
  4. Type “zeek” for Index Name and click Save to create your new index.

Install and configure the Corelight For Splunk app

The Corelight For Splunk app is created by the Zeek team directly for use with Corelight (enterprise Zeek) and open-source Zeek sensors.  We’ll use this app to help parse, index, and visualize Zeek logs.

Note that Splunk has also published their own Splunk Add-on for Zeek aka Bro app which helps to ingest Zeek logs but does not feature any sort of dashboards or reports.

  1. Download and install the Corelight for Splunk app onto your Splunk server.  This can either be done within the Splunk server itself or by manually downloading and installing as you would all other Splunk apps.
  2. You can navigate to the app to verify that it is installed correctly.  However, since we have not yet configured our sensor to send data, the dashboards will be blank.
  3. In the top right menu navigate to Settings -> Knowledge -> Event types.
  4. In the App dropdown menu, select Corelight For Splunk and click on corelight_idx.
  5. In the Search string field type index=zeek.  This tells the Corelight for Splunk app to search for data in the “zeek” index we created earlier.

Create a splunk user to run the Splunk Universal Forwarder

Back in the Zeek sensor, create a splunk user and add it to the bro group.

sudo useradd splunk -g bro

Install and configure a Splunk Universal Forwarder

  1. Login to your Splunk account and download the latest Splunk Universal Forwarder.  Once logged in, click “Download Now” for the Linux 64-bit .rpm installer.  Note that Splunk also generates a convenient wget command you can use from the sensor itself once you accept the license agreement. As of this writing, the latest release is version 7.2.3.  If the download URL referenced in the wget command below no longer works, download directly as noted above.
    wget -O splunkforwarder-7.2.3-06d57c595b80-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.3&product=universalforwarder&filename=splunkforwarder-7.2.3-06d57c595b80-linux-2.6-x86_64.rpm&wget=true'
  2. Install the forwarder to /opt/splunkforwarder using the rpm command.
    sudo rpm -i splunkforwarder-<…>-linux-2.6-x86_64.rpm
  3. Start the forwarder to accept the license agreement and create an administrative password.
    cd /opt/splunkforwarder/bin
    ./splunk start --accept-license
  4. Stop the forwarder.
    ./splunk stop
  5. Remove the default data processing limit.  Edit /opt/splunkforwarder/etc/system/local/limits.conf and add the following lines.Note that given the volume of data that Zeek generates, the forwarder may never process all log data if the default limit is not removed.
    [thruput]
    maxKBps = 0 # means unlimited
    
  6. Edit /opt/splunkforwarder/etc/system/local/inputs.conf to monitor desired Zeek logs.  An example inputs.conf is below but may or may not include the logs you wish to ingest. Note that the index and sourcetype fields are leveraging the “zeek” naming scheme to match the “zeek” index we created in Splunk.
    [default]
    host = sensor
    
    [monitor:///opt/bro/logs/current/conn.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_conn
    
    [monitor:///opt/bro/logs/current/dns.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_dns
    
    [monitor:///opt/bro/logs/current/software.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_software
    
    [monitor:///opt/bro/logs/current/smtp.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_smtp
    
    [monitor:///opt/bro/logs/current/ssl.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_ssl
    
    [monitor:///opt/bro/logs/current/ssh.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_ssh
    
    [monitor:///opt/bro/logs/current/x509.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_x509
    
    [monitor:///opt/bro/logs/current/ftp.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_ftp
    
    [monitor:///opt/bro/logs/current/http.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_http
    
    [monitor:///opt/bro/logs/current/rdp.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_rdp
    
    [monitor:///opt/bro/logs/current/smb_files.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_smb_files
    
    [monitor:///opt/bro/logs/current/smb_mapping.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_smb_mapping
    
    [monitor:///opt/bro/logs/current/snmp.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_snmp
    
    [monitor:///opt/bro/logs/current/sip.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_sip
    
    [monitor:///opt/bro/logs/current/files.log]
    _TCP_ROUTING = *
    index = zeek
    sourcetype = corelight_files
    
  7. Edit /opt/splunkforwarder/etc/system/local/outputs.conf to send data to your Splunk server.  In the sample file below, replace each instance of splunkserver:9997 with your own server name/IP and port number.
    [tcpout]
    defaultGroup = default-autolb-group
    
    [tcpout:default-autolb-group]
    server = splunkserver:9997
    
    [tcpout-server://splunkserver:9997]
    
  8. Make the splunk user and bro group the owner of the entire /opt/splunkforwarder directory.
    sudo chown -R splunk:bro /opt/splunkforwarder/
  9. Start the forwarder as the splunk user and confirm it successfully ran.  You can check /opt/splunkforwarder/var/log/splunk/splunkd.log for any issues.
    su splunk
    cd /opt/splunkforwarder/bin
    ./splunk start

Up Next

In Part III of this series, we will dig into Zeek data with sample Splunk queries.

Zeekurity Zen: Part I – How to Install Zeek (Bro) on CentOS 7

This is part of a larger series on building a Zeek (Bro) network sensor.

Overview

Zeek (formerly named Bro) is my favorite security monitoring platform, and I’ve used and promoted it throughout my career.  It generates rich network metadata that’s incredibly valuable for incident response, forensics, and general troubleshooting.

Perhaps the main challenge with using Zeek is actually setting it up.  While today there exists Corelight (an easy-to-use Zeek appliance with enterprise support), not everyone has the budget for something like this.  This series will walkthrough Zeek setup, integration with Splunk, and various tips and tricks I’ve learned over the years.

In Part I, we’ll walkthrough several steps:

  1. Disabling NetworkManager and instead using the “network” service
  2. Disabling NIC offloading functions to ensure Zeek sees full packet data and minimizes packet loss.
  3. Enabling the “network” service to apply network sniffing optimizations.
  4. Setting interfaces to promiscuous mode to ensure all packets are captured and analyzed.
  5. Install PF_RING and libmaxminddb to improve packet capture performance and enable IP geolocation capability.
  6. Build Zeek from source with optimizations.
  7. Create a non-root Zeek user to minimize impact in the event that Zeek is compromised.
  8. Deploy and run Zeek to start analyzing traffic.
  9. Create a cron job to perform Zeek maintenance tasks.

Disable the NetworkManager service

  1. Stop and disable the NetworkManager service.
    sudo systemctl stop NetworkManager
    sudo systemctl disable NetworkManager
    Removed symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
  2. Verify that NetworkManager has been disabled.
    sudo systemctl list-unit-files | grep NetworkManager
    NetworkManager-dispatcher.service             disabled
    NetworkManager-wait-online.service            enabled
    NetworkManager.service                        disabled

Disable NIC offloading functions

  1. Use ethtool to determine the maximum ring parameters for your sniffing interfaces.  The example below assumes an interface named ens34.
    sudo ethtool -g ens34
    Ring parameters for ens34:
    Pre-set maximums:
    RX:             4096
    RX Mini:        0
    RX Jumbo:       0
    TX:             4096
    Current hardware settings:
    RX:             256
    RX Mini:        0
    RX Jumbo:       0
    TX:             256
  2. As root/sudo, edit the /etc/sysconfig/network-scripts/ifcfg-<sniffinginterface> file for each sniffing network interface and change or add the following lines. Respectively, each line will disable control from NetworkManager, disable DHCP, and add appropriate ethtool options. Note that after “rx” you want to enter the maximum ring parameter as determined in the step above.
    NM_CONTROLLED=no
    BOOTPROTO=none
    ONBOOT=yes
    IPV6INIT=no
    ETHTOOL_OPTS="-G ${DEVICE} rx <max ring parameter determined from step 1 above>; -K ${DEVICE} rx off; -K ${DEVICE} tx off; -K ${DEVICE} sg off; -K ${DEVICE} tso off; -K ${DEVICE} ufo off; -K ${DEVICE} gso off; -K ${DEVICE} gro off; -K ${DEVICE} lro off"
  3. Your file should now look something like this.
    TYPE=Ethernet
    BOOTPROTO=none
    DEFROUTE=yes
    IPV4_FAILURE_FATAL=no
    IPV6INIT=no
    IPV6_AUTOCONF=yes
    IPV6_DEFROUTE=yes
    IPV6_PEERDNS=yes
    IPV6_PEERROUTES=yes
    IPV6_FAILURE_FATAL=no
    NAME=ens34
    UUID=b22f5d92-3f1e-430b-b660-cb9376d8c0c0
    DEVICE=ens34
    ONBOOT=yes
    PEERDNS=yes
    PEERROUTES=yes
    USERS=root
    NM_CONTROLLED=no
    ETHTOOL_OPTS="-G ${DEVICE} rx 4096; -K ${DEVICE} rx off; -K ${DEVICE} tx off; -K ${DEVICE} sg off; -K ${DEVICE} tso off; -K ${DEVICE} ufo off; -K ${DEVICE} gso off; -K ${DEVICE} gro off; -K ${DEVICE} lro off"

Enable “network” service

  1. As root/sudo, edit /etc/resolv.conf in your favorite text editor to add your DNS server(s) and then save the file.
    nameserver aaa.bbb.ccc.ddd
    nameserver www.xxx.yyy.zzz
  2. Still as root/sudo, enable the “network” service.
    sudo systemctl enable network
  3. Finally, restart the “network” service.
    sudo systemctl restart network

Set sniffing network interfaces to promiscuous mode

  1. As root/sudo, edit /etc/rc.d/rc.local in your favorite text editor.
  2. Add the following line for each sniffing interface, assuming ens34 is your sniffing interface.
    ip link set ens34 promisc on
  3. Save the file and run the following commands to make the changes permanent and start on boot.
    sudo chmod u+x /etc/rc.d/rc.local
    sudo systemctl enable rc-local
    sudo systemctl start rc-local
  4. Reboot your system and verify all the changes made thus far have persisted. Verify that PROMISC is listed in the network interface status.
    ip a show ens34 | grep -i promisc
    3: ens34: < BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP > mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

Install Zeek Dependencies

  1. Run the following yum command to download the required dependencies.
    sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python-devel swig zlib-devel gperftools jemalloc-devel kernel-devel kernel-headers
  2. Ensure all your packages are up to date and reboot your system.
    sudo yum update
    sudo reboot

Install PF_RING

PF_RING significantly improves packet capture performance and allows for high speed capture on commodity hardware.

  1. Download PF_RING into /usr/src and compile the PF_RING libraries. Below, I’ve chosen to the use the latest stable (as of this writing) version.
    cd /usr/src
    sudo wget https://github.com/ntop/PF_RING/archive/7.2.0-stable.zip
    sudo unzip 7.2.0-stable.zip
  2. Next, compile the PF_RING libraries, PF_RING-enabled libpcap, PF_RING-enabled tcpdump, and the PF_RING kernel module.  Replace the version number placeholders for PF_RING (X.Y.Z) and tcpdump (A.B.C) with whichever versions you’re working with.  Including the PF_RING version in the /opt/pfring-X.Y.Z directory name will help you easily identify which version of PF_RING you’re currently running.
    cd PF_RING-X.Y.Z-stable/userland/lib
    sudo ./configure --prefix=/opt/pfring-X.Y.Z
    sudo make
    sudo make install
    
    cd ../libpcap
    sudo ./configure --prefix=/opt/pfring-X-Y-Z
    sudo make
    sudo make install
    
    cd ../tcpdump-A.B.C
    sudo ./configure --prefix=/opt/pfring-X-Y-Z
    sudo make
    sudo make install
    
    cd ../../kernel
    sudo make
    sudo make install

    If you run into errors compiling the kernel module, make sure your system is up to date and then try rebooting. Typically, the kernel section will fail if the kernel headers you’re compiling against are not the same version as the current running kernel.

    sudo yum update
    sudo reboot
  3. Test that the module was compiled correctly by loading it with modprobe.
    sudo modprobe pf_ring enable_tx_capture=0 min_num_slots=65534
  4. Assuming that the module was loaded correctly, you can view module details in /proc/net/pf_ring/info.
    cat /proc/net/pf_ring/info
    
    PF_RING Version          : 7.2.0 (unknown)
    Total rings              : 0
    
    Standard (non ZC) Options
    Ring slots               : 65534
    Slot version             : 17
    Capture TX               : No [RX only]
    IP Defragment            : No
    Socket Mode              : Standard
    Cluster Fragment Queue   : 0
    Cluster Fragment Discard : 0
  5. Running the modprobe command above does not persist through reboots.  To load the PF_RING module at boot time, we will create two files with the following content. Note that these must be created as root/sudo.

    /etc/modules-load.d/pf_ring.conf

    # load pf_ring
    pf_ring

    /etc/modprobe.d/pf_ring.conf

    options pf_ring enable_tx_capture=0 min_num_slots=65534
  6. Reboot the server and verify that the /proc/net/pf_ring/info file exists and shows the same details as noted in step 4 above.
    sudo reboot
    cat /proc/net/pf_ring/info
    
    PF_RING Version          : 7.2.0 (unknown)
    Total rings              : 0
    
    Standard (non ZC) Options
    Ring slots               : 65534
    Slot version             : 17
    Capture TX               : No [RX only]
    IP Defragment            : No
    Socket Mode              : Standard
    Cluster Fragment Queue   : 0
    Cluster Fragment Discard : 0

Configure GeoIP Support

  1. Install the libmaxminddb development library from the EPEL repo.
    sudo yum --enablerepo=extras install epel-release
    sudo yum install libmaxminddb-devel
  2. Download and untar the GeoLite2 database.
    wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz
    tar xzvf GeoLite2-City.tar.gz
  3. Move the GeoLite2-City.mmdb file in the extracted GeoLite2-City_YYYYMMDD directory to /usr/share/GeoIP.
    sudo mv GeoLite2-City_YYYYMMDD/GeoLite2-City.mmdb /usr/share/GeoIP/GeoLite2-City.mmdb

Download, Compile, and Install Zeek

We will configure Zeek to install in the /opt/bro directory, using our PF_RING module, and with jemalloc (for improved performance).

As of this writing, the latest release is version 2.6.1.  If the download URL referenced in the wget command below no longer works, you can download the latest stable release directly from the Zeek download page.

cd
wget https://www.zeek.org/downloads/bro-2.6.1.tar.gz
tar -xzvf bro-2.6.1.tar.gz
cd bro-2.6.1
./configure --prefix=/opt/bro --with-pcap=/opt/pfring-7.2.0/ --enable-jemalloc
sudo make
sudo make install

Create a non-root user to run Zeek

  1. Create a bro user and add it to the bro group.
    sudo groupadd bro
    sudo useradd bro -g bro
  2. Give the Zeek binaries permissions to capture packets.
    sudo setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/bro
    sudo setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/broctl
  3. Set the bro user as owner of the bro directory.
    sudo chown -R bro:bro /opt/bro

Add Zeek to PATH

As root/sudo, create /etc/profile.d/bro.sh and add the following.

pathmunge /opt/bro/bin

Configure Zeek

  1. Edit /opt/bro/etc/node.cfg to configure the number of nodes and to leverage PF_RING.  It is recommended to use a maximum of one or two less workers than the total number of CPU cores available on your sensor.  In the example configuration below we are configuring a total of two workers, load balanced across two CPUs.
    # Example BroControl node configuration.
    # Below is an example configuration of standalone sensor leveraging PF_RING.
    
    [logger]
    type=logger
    host=localhost
    
    [manager]
    type=manager
    host=localhost
    
    [proxy-1]
    type=proxy
    host=localhost
    
    [worker-1]
    type=worker
    host=localhost
    interface=ens34
    lb_method=pf_ring
    lb_procs=2
    pin_cpus=0,1
    
    [worker-2]
    type=worker
    host=localhost
    interface=ens34
    lb_method=pf_ring
    lb_procs=2
    pin_cpus=0,1
  2. Edit /opt/bro/share/bro/site/local.bro to enable or disable scripts as needed.

Start Zeek

  1. Switch to the non-root bro user.
    sudo su bro
  2. As the bro user, run broctl deploy to apply configurations and run Bro.
    broctl deploy
    checking configurations ...
    installing ...
    removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/site ...
    removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/auto ...
    creating policy directories ...
    installing site policies ...
    generating cluster-layout.bro ...
    generating local-networks.bro ...
    generating broctl-config.bro ...
    generating broctl-config.sh ...
    stopping ...
    stopping workers ...
    stopping proxy ...
    stopping manager ...
    stopping logger ...
    starting ...
    starting logger ...
    starting manager ...
    starting proxy ...
    starting workers ...
  3. If your output looks similar to what’s shown above, you should be good to go. To verify Zeek is running successfully, you can run broctl status.
    broctl status
    Name         Type    Host             Status    Pid    Started
    logger       logger  localhost        running   1774   10 Oct 23:15:31
    manager      manager localhost        running   1820   10 Oct 23:15:32
    proxy-1      proxy   localhost        running   1865   10 Oct 23:15:33
    worker-1-1   worker  localhost        running   1950   10 Oct 23:15:35
    worker-1-2   worker  localhost        running   1951   10 Oct 23:15:35
    worker-2-1   worker  localhost        running   1955   10 Oct 23:15:35
    worker-2-2   worker  localhost        running   1954   10 Oct 23:15:35
  4. You should now see logs being generated in /opt/bro/logs/current.
    ls -l
    total 2276
    -rw-rw-r--. 1 bro bro   1573 Oct 10 23:15 broker.log
    -rw-rw-r--. 1 bro bro    593 Oct 10 23:45 capture_loss.log
    -rw-rw-r--. 1 bro bro   1970 Oct 10 23:15 cluster.log
    -rw-rw-r--. 1 bro bro 673435 Oct 10 23:52 conn.log
    -rw-rw-r--. 1 bro bro 580865 Oct 10 23:52 dns.log
    -rw-rw-r--. 1 bro bro   3830 Oct 10 23:49 dpd.log
    -rw-rw-r--. 1 bro bro   1406 Oct 10 23:47 files.log
    -rw-rw-r--. 1 bro bro  26108 Oct 10 23:48 http.log
    -rw-rw-r--. 1 bro bro  24646 Oct 10 23:15 loaded_scripts.log
    -rw-rw-r--. 1 bro bro    753 Oct 10 23:18 notice.log
    -rw-rw-r--. 1 bro bro    187 Oct 10 23:15 packet_filter.log
    -rw-rw-r--. 1 bro bro    743 Oct 10 23:46 software.log
    -rw-rw-r--. 1 bro bro  86512 Oct 10 23:51 ssl.log
    -rw-rw-r--. 1 bro bro   5446 Oct 10 23:50 stats.log
    -rw-rw-r--. 1 bro bro      0 Oct 10 23:15 stderr.log
    -rw-rw-r--. 1 bro bro    188 Oct 10 23:15 stdout.log
    -rw-rw-r--. 1 bro bro 240866 Oct 10 23:51 weird.log
  5. If you’re running into issues, broctl diag can provide more detailed output for troubleshooting purposes.
    broctl diag

BroControl Cron

BroControl features a cron command to check for and restart crashed nodes and to perform other maintenance tasks.  To take advantage of this, let’s set up a cron job.

  1. Edit the crontab of the non-root bro user.
    crontab -e
  2. Add the following to set up a cron job that runs every five minutes.  You can adjust the frequency to your liking.
    */5 * * * * /opt/bro/bin/broctl cron

Up Next

In Part II of this series, we will walkthrough how to send Zeek logs to Splunk and take advantage of the Corelight For Splunk app.

References

Adding a path in CentOS: https://serverfault.com/questions/102932/adding-a-directory-to-path-in-centos
Zeek official documentationhttps://www.zeek.org/documentation/index.html
Disabling NetworkManager: https://www.thegeekdiary.com/centos-rhel-7-how-to-disable-networkmanager
NIC offloadinghttps://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html