Building Bro: Part I – How to Install Bro/Zeek on CentOS 7

This is part of a larger series on building a Bro/Zeek network sensor.

Overview

Bro (now renamed Zeek) is my favorite security monitoring platform, and I’ve used and promoted it throughout my career.  It generates rich network metadata that’s incredibly valuable for incident response, forensics, and general troubleshooting.
Perhaps the main challenge with using Bro is actually setting it up.  While today there exists Corelight (an easy-to-use Bro appliance with enterprise support), not everyone has the budget for something like this.  This series will walkthrough Bro setup, integration with Splunk, and various tips and tricks I’ve learned over the years.
In Part I, we’ll walkthrough several steps:
  1. Disabling NetworkManager and instead using the “network” service to apply network sniffing optimizations.
  2. Disabling NIC offloading functions to ensure Bro sees full packet data and minimizes packet loss.
  3. Setting interfaces to promiscuous mode to ensure all packets are captured and analyzed.
  4. Install PF_RING and libmaxminddb to improve packet capture performance and enable IP geolocation capability.
  5. Build Bro from source with optimizations.
  6. Create a non-root Bro user to minimize impact in the event that Bro is compromised.
  7. Deploy and run Bro to start analyzing traffic.
  8. Create a cron job to perform Bro maintenance tasks.

Disable the NetworkManager service and enable the “network” service

  1. Stop and disable the NetworkManager service.
    sudo systemctl stop NetworkManager
    sudo systemctl disable NetworkManager
    Removed symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
  2. Verify that NetworkManager has been disabled.
    sudo systemctl list-unit-files | grep NetworkManager
    NetworkManager-dispatcher.service             disabled
    NetworkManager-wait-online.service            enabled
    NetworkManager.service                        disabled

Disable NIC offloading functions

  1. Use ethtool to determine the maximum ring parameters for your sniffing interfaces.  The example below assumes an interface named ens34.
    sudo ethtool -g ens34
    Ring parameters for ens34:
    Pre-set maximums:
    RX:             4096
    RX Mini:        0
    RX Jumbo:       0
    TX:             4096
    Current hardware settings:
    RX:             256
    RX Mini:        0
    RX Jumbo:       0
    TX:             256
  2. As root/sudo, edit the /etc/sysconfig/network-scripts/ifcfg-<sniffinginterface> file for each sniffing network interface and change or add the following lines. Respectively, each line will disable control from NetworkManager, disable DHCP, and add appropriate ethtool options. Note that after “rx” you want to enter the maximum ring parameter as determined in the step above.
    NM_CONTROLLED=no
    BOOTPROTO=none
    ONBOOT=yes
    IPV6INIT=no
    ETHTOOL_OPTS="-G ${DEVICE} rx <max ring parameter determined from step 1 above>; -K ${DEVICE} rx off; -K ${DEVICE} tx off; -K ${DEVICE} sg off; -K ${DEVICE} tso off; -K ${DEVICE} ufo off; -K ${DEVICE} gso off; -K ${DEVICE} gro off; -K ${DEVICE} lro off"
  3. Your file should now look something like this.
    TYPE=Ethernet
    BOOTPROTO=none
    DEFROUTE=yes
    IPV4_FAILURE_FATAL=no
    IPV6INIT=no
    IPV6_AUTOCONF=yes
    IPV6_DEFROUTE=yes
    IPV6_PEERDNS=yes
    IPV6_PEERROUTES=yes
    IPV6_FAILURE_FATAL=no
    NAME=ens34
    UUID=b22f5d92-3f1e-430b-b660-cb9376d8c0c0
    DEVICE=ens34
    ONBOOT=yes
    PEERDNS=yes
    PEERROUTES=yes
    USERS=root
    NM_CONTROLLED=no
    ETHTOOL_OPTS="-G ${DEVICE} rx 4096; -K ${DEVICE} rx off; -K ${DEVICE} tx off; -K ${DEVICE} sg off; -K ${DEVICE} tso off; -K ${DEVICE} ufo off; -K ${DEVICE} gso off; -K ${DEVICE} gro off; -K ${DEVICE} lro off"

Set sniffing network interfaces to promiscuous mode

  1. As root/sudo, edit /etc/rc.d/rc.local in your favorite text editor.
  2. Add the following line for each sniffing interface, assuming ens34 is your sniffing interface.
    ip link set ens34 promisc on
  3. Save the file and run the following commands to make the changes permanent and start on boot.
    sudo chmod u+x /etc/rc.d/rc.local
    sudo systemctl enable rc-local
    sudo systemctl start rc-local
  4. Reboot your system and verify all the changes made thus far have persisted. Verify that PROMISC is listed in the network interface status.
    ip a show ens34 | grep -i promisc
    3: ens34: < BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP > mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

Install Bro Dependencies

  1. Run the following yum command to download the required dependencies.
    sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python-devel swig zlib-devel gperftools jemalloc-devel kernel-devel kernel-headers
  2. Ensure all your packages are up to date and reboot your system.
    sudo yum update
    sudo reboot

Install PF_RING

PF_RING significantly improves packet capture performance and allows for high speed capture on commodity hardware.

  1. Download PF_RING into /usr/src and compile the PF_RING libraries. Below, I’ve chosen to the use the latest stable (as of this writing) version.
    cd /usr/src
    sudo wget https://github.com/ntop/PF_RING/archive/7.2.0-stable.zip
    sudo unzip 7.2.0-stable.zip
  2. Next, compile the PF_RING libraries, PF_RING-enabled libpcap, PF_RING-enabled tcpdump, and the PF_RING kernel module.  Replace the version number placeholders for PF_RING (X.Y.Z) and tcpdump (A.B.C) with whichever versions you’re working with.  Including the PF_RING version in the /opt/pfring-X.Y.Z directory name will help you easily identify which version of PF_RING you’re currently running.
    cd PF_RING-X.Y.Z-stable/userland/lib
    sudo ./configure --prefix=/opt/pfring-X.Y.Z
    sudo make
    sudo make install
    
    cd ../libpcap
    sudo ./configure --prefix=/opt/pfring-X-Y-Z
    sudo make
    sudo make install
    
    cd ../tcpdump-A.B.C
    sudo ./configure --prefix=/opt/pfring-X-Y-Z
    sudo make
    sudo make install
    
    cd ../../kernel
    sudo make
    sudo make install

    If you run into errors compiling the kernel module, make sure your system is up to date and then try rebooting. Typically, the kernel section will fail if the kernel headers you’re compiling against are not the same version as the current running kernel.

    sudo yum update
    sudo reboot
  3. Test that the module was compiled correctly by loading it with modprobe.
    sudo modprobe pf_ring enable_tx_capture=0 min_num_slots=65534
  4. Assuming that the module was loaded correctly, you can view module details in /proc/net/pf_ring/info.
    cat /proc/net/pf_ring/info
    
    PF_RING Version          : 7.2.0 (unknown)
    Total rings              : 0
    
    Standard (non ZC) Options
    Ring slots               : 65534
    Slot version             : 17
    Capture TX               : No [RX only]
    IP Defragment            : No
    Socket Mode              : Standard
    Cluster Fragment Queue   : 0
    Cluster Fragment Discard : 0
  5. Running the modprobe command above does not persist through reboots.  To load the PF_RING module at boot time, we will create two files with the following content. Note that these must be created as root/sudo.

    /etc/modules-load.d/pf_ring.conf

    # load pf_ring
    pf_ring

    /etc/modprobe.d/pf_ring.conf

    options pf_ring enable_tx_capture=0 min_num_slots=65534
  6. Reboot the server and verify that the /proc/net/pf_ring/info file exists and shows the same details as noted in step 4 above.
    sudo reboot
    cat /proc/net/pf_ring/info
    
    PF_RING Version          : 7.2.0 (unknown)
    Total rings              : 0
    
    Standard (non ZC) Options
    Ring slots               : 65534
    Slot version             : 17
    Capture TX               : No [RX only]
    IP Defragment            : No
    Socket Mode              : Standard
    Cluster Fragment Queue   : 0
    Cluster Fragment Discard : 0

Configure GeoIP Support

  1. Install the libmaxminddb development library from the EPEL repo.
    sudo yum --enablerepo=extras install epel-release
    sudo yum install libmaxminddb-devel
  2. Download and untar the GeoLite2 database.
    wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz
    tar xzvf GeoLite2-City.tar.gz
  3. Move the GeoLite2-City.mmdb file in the extracted GeoLite2-City_YYYYMMDD directory to /usr/share/GeoIP.
    sudo mv GeoLite2-City_YYYYMMDD/GeoLite2-City.mmdb /usr/share/GeoIP/GeoLite2-City.mmdb

Download, Compile, and Install Bro

We will configure Bro to install in the /opt/bro directory, using our PF_RING module, and with jemalloc (for improved performance).

Note that in the step below we are cloning from Bro’s master git which is the current development version.  As of this writing, this will become the eventual 2.6 release.  If you are looking to place this sensor in production, it is recommended you leverage the latest stable release directly from the Bro download page.

cd
git clone --recursive git://git.bro.org/bro
cd bro
./configure --prefix=/opt/bro --with-pcap=/opt/pfring-7.2.0/ --enable-jemalloc
make
make install

Create a non-root user to run Bro

  1. Create a bro user and add it to the bro group.
    sudo groupadd bro
    sudo useradd bro -g bro
  2. Give the Bro binaries permissions to capture packets.
    sudo setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/bro
    sudo setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/broctl
  3. Set the bro user as owner of the bro directory.
    sudo chown -R bro:bro /opt/bro

Add Bro to PATH

As root/sudo, create /etc/profile.d/bro.sh and add the following.
pathmunge /opt/bro/bin

Configure Bro

  1. Edit /opt/bro/etc/node.cfg to configure the number of nodes and to leverage PF_RING.  It is recommended to use a maximum of one or two less workers than the total number of CPU cores available on your sensor.  In the example configuration below we are configuring a total of two workers, load balanced across two CPUs.
    # Example BroControl node configuration.
    # Below is an example configuration of standalone sensor leveraging PF_RING.
    
    [logger]
    type=logger
    host=localhost
    
    [manager]
    type=manager
    host=localhost
    
    [proxy-1]
    type=proxy
    host=localhost
    
    [worker-1]
    type=worker
    host=localhost
    interface=ens34
    lb_method=pf_ring
    lb_procs=2
    pin_cpus=0,1
    
    [worker-2]
    type=worker
    host=localhost
    interface=ens34
    lb_method=pf_ring
    lb_procs=2
    pin_cpus=0,1
  2. Edit /opt/bro/share/bro/site/local.bro to enable or disable scripts as needed.

Start Bro

    1. Switch to the non-root bro user.
      sudo su bro
    2. As the bro user, run broctl deploy to apply configurations and run Bro.
      broctl deploy
      checking configurations ...
      installing ...
      removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/site ...
      removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/auto ...
      creating policy directories ...
      installing site policies ...
      generating cluster-layout.bro ...
      generating local-networks.bro ...
      generating broctl-config.bro ...
      generating broctl-config.sh ...
      stopping ...
      stopping workers ...
      stopping proxy ...
      stopping manager ...
      stopping logger ...
      starting ...
      starting logger ...
      starting manager ...
      starting proxy ...
      starting workers ...
    3. If your output looks similar to what’s shown above, you should be good to go. To verify Bro is running successfully, you can run broctl status.
      broctl status
      Name         Type    Host             Status    Pid    Started
      logger       logger  localhost        running   1774   10 Oct 23:15:31
      manager      manager localhost        running   1820   10 Oct 23:15:32
      proxy-1      proxy   localhost        running   1865   10 Oct 23:15:33
      worker-1-1   worker  localhost        running   1950   10 Oct 23:15:35
      worker-1-2   worker  localhost        running   1951   10 Oct 23:15:35
      worker-2-1   worker  localhost        running   1955   10 Oct 23:15:35
      worker-2-2   worker  localhost        running   1954   10 Oct 23:15:35
    4. You should now see logs being generated in /opt/bro/logs/current.
      ls -l
      total 2276
      -rw-rw-r--. 1 bro bro   1573 Oct 10 23:15 broker.log
      -rw-rw-r--. 1 bro bro    593 Oct 10 23:45 capture_loss.log
      -rw-rw-r--. 1 bro bro   1970 Oct 10 23:15 cluster.log
      -rw-rw-r--. 1 bro bro 673435 Oct 10 23:52 conn.log
      -rw-rw-r--. 1 bro bro 580865 Oct 10 23:52 dns.log
      -rw-rw-r--. 1 bro bro   3830 Oct 10 23:49 dpd.log
      -rw-rw-r--. 1 bro bro   1406 Oct 10 23:47 files.log
      -rw-rw-r--. 1 bro bro  26108 Oct 10 23:48 http.log
      -rw-rw-r--. 1 bro bro  24646 Oct 10 23:15 loaded_scripts.log
      -rw-rw-r--. 1 bro bro    753 Oct 10 23:18 notice.log
      -rw-rw-r--. 1 bro bro    187 Oct 10 23:15 packet_filter.log
      -rw-rw-r--. 1 bro bro    743 Oct 10 23:46 software.log
      -rw-rw-r--. 1 bro bro  86512 Oct 10 23:51 ssl.log
      -rw-rw-r--. 1 bro bro   5446 Oct 10 23:50 stats.log
      -rw-rw-r--. 1 bro bro      0 Oct 10 23:15 stderr.log
      -rw-rw-r--. 1 bro bro    188 Oct 10 23:15 stdout.log
      -rw-rw-r--. 1 bro bro 240866 Oct 10 23:51 weird.log
    5. If you’re running into issues, broctl diag can provide more detailed output for troubleshooting purposes.
      broctl diag

BroControl Cron

BroControl features a cron command to check for and restart crashed nodes and to perform other maintenance tasks.  To take advantage of this, let’s set up a cron job.

  1. Edit the crontab of the non-root bro user.
    crontab -e
  2. Add the following to set up a cron job that runs every five minutes.  You can adjust the frequency to your liking.
    */5 * * * * /opt/bro/bin/broctl cron

Up Next

In Part II of this series, we will walkthrough how to send Bro logs to Splunk and take advantage of the Corelight For Splunk app.

References

Adding a path in CentOS: https://serverfault.com/questions/102932/adding-a-directory-to-path-in-centos
Bro official documentationhttps://www.bro.org/sphinx-git/index.html
Disabling NetworkManager: https://www.thegeekdiary.com/centos-rhel-7-how-to-disable-networkmanager
NIC offloadinghttps://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html

Facebooktwittergoogle_plusredditpinterestlinkedintumblrmailFacebooktwittergoogle_plusredditpinterestlinkedintumblrmail

How To Build a SANS GIAC Index

Over the years, my most popular article has been about how to successfully pass SANS GIAC exams.  Most people focus on my second recommendation on building an index for the course material.  Unfortunately, the index guide I linked to is no longer available.  Since I’ve received a number of requests for the Word doc, I’ve decided to write my own version of how to do this.  Enjoy!

  1. Create a spreadsheet with tabs labeled for each book in the course.  For example, “503.1”, “503.2 + 503.3”, etc.
  2. Label the first four columns with: “Page”, “Keyword 1”, “Keyword 2”, and “Keyword 3”.
  3. Read through each course book and summarize each page into three keywords or phrases (e.g. “tcpdump output overview”, “nibble definition”, etc) and note these in your spreadsheet.  Skip this for any pages with no notable information.
  4. For each tab in the spreadsheet, insert a new column before column A and title it “Book”.  For each existing row of pages, populate the book number.
  5. Copy columns A (“Book”) and B (“Page”) and insert both of them before the columns for “Keyword 2” and “Keyword 3”.
  6. Cut the “Book”, “Page”, and “Keyword 3” block and paste them below their respective columns in A, B, and C.  Repeat this for the “Book”, “Page”, and “Keyword 2” block and place these under the “Keyword 3” block you just pasted.
  7. Repeat steps 4 – 6 for each tab in your spreadsheet and remember to save your work. 🙂
  8. Create a new tab in the spreadsheet titled, “Complete”.  This will contain all the data from the previous tabs.
  9. Copy and paste columns A, B, and C from each of the previous tabs into columns A, B, and C of the “Complete” tab.  Paste each new tab’s data below the previous.
  10. In the “Complete” tab, move column C (“Keyword 1”) before column A (“Book”).
  11. Sort the new column A (“Keyword”) alphabetically and perform any edits and formatting cleanup (e.g. remove rows with column headers due to pasting and any blank rows).  Congratulations, you have an index!

Hope this helps and best of luck on the exam!  You’ll do great. 🙂

One final note.  Please don’t ask for the indexes I created, as I will not be sharing them.  The whole point in building your own index is so you’ll learn and retain the material.  Asking for mine or taking someone else’s is a shortcut that will likely lead to your own disappointment come exam time. 😛

Facebooktwittergoogle_plusredditpinterestlinkedintumblrmailFacebooktwittergoogle_plusredditpinterestlinkedintumblrmail

How to determine your Ring Doorbell Pro firmware version

I have a love/hate relationship with my Ring Doorbell.  When I purchased it in 2016 it worked great for a year with minimal issues.  As it became more popular, I noticed the quality dropped with video freezes, black videos, and missed motion events.  This led me to the Ring Doorbell subreddit where I found a community of users who were also experiencing the same issues.  This made me feel a bit better (misery loves company, right?) but still disappointed that this $250 doorbell no longer lived up to its promise.

A number of the users who shared their negative feedback traced some of the poor service to firmware changes.  It soon became commonplace to post your issues along with the firmware version your device was currently running.  Pretty basic troubleshooting practice.  So it was much to everyone’s dismay when in late 2017, Ring decided to change how they displayed firmware versions.  In short, if your device is on the latest version the mobile app would only display, “Up to Date” as opposed to an actual firmware version number.  But without an actual firmware version number to compare with others, for all you know your device may actually be on an older version but hasn’t properly updated itself such that it merely *thinks* it is up to date.  Presumably, if it was actually out of date, it will display a version number, but this is useless as you cannot manually force an upgrade.  And again, you can’t compare with others to know which version you should actually be on.  This also makes it difficult to track changes that Ring is making and correlate them to your device’s performance improvements or degradations.  As you might imagine, there was a lively Reddit discussion on this.

Being in the information security field, I know that software version numbers are critical to confirming that my application is fully patched against any identified security vulnerabilities.  Naturally, I was disappointed by this change and soon looked for ways to determine the version number using, what else?  My Palo Alto firewall.

I knew my Ring Doorbell had to communicate with Ring’s servers in some way to check if it was running the latest firmware version.  I figured an easy way for Ring to do this is via user agent strings.  So I first checked the Monitor tab to see if the user agent of the device appeared in the URL Filtering view.  Sadly, the user agent field was blank suggesting that it wasn’t normal http traffic that this information was in.  Still feeling confident that the user agent must be somewhere, I decided to run a packet capture through the Palo Alto via Monitor -> Packet Capture.

  1. Navigate to Configure Filtering -> Manage Filters.
  2. Click Add and configure the Source with the IP address of your Ring Doorbell.  I have mine statically assigned via my DHCP server but this should be fairly easy for you to determine either in your wireless router or your Palo Alto firewall.  Click OK when done.

3. Back in the Configure Filtering menu turn Filtering to ON.

4. In Configure Capturing click Add and select firewall for Stage and give your packet capture a file name. In this example I’ve used ringdoorbell.  Click OK and your view should look like the one below.

5. Once you’re ready, in the same view set Packet Capture to ON.  You’ll receive a warning about packet captures degrading system performance and to remember to disable the feature once you’re done.  Click OK to proceed.

6. Now we need to generate some traffic through the doorbell to hopefully find the user agent string in the packet captures.  Start a Live View session through your Ring mobile app and let it run for at least 30 seconds.  Once completed, set Packet Capture back to OFF.

7. Click the Refresh button in the top right corner or reload the page to find your newly created packet capture file in the Captured Files view.  Click on it to download the file to your computer.

8. You’ve got a few options to view this file.  Since I’m on a MacBook Pro, I’ll walk through how to use tcpdump to quickly find the user agent.  You could also use Wireshark to accomplish this.

Locate the file on your system and use the following tcpdump command: tcpdump -nn -r ringdoorbell.pcap -A | grep -i agent

tcpdump -nn -r ringdoorbell.pcap -A | grep -i agent
reading from file ringdoorbell.pcap, link-type EN10MB (Ethernet)
User-Agent: Device/lpdv2/1.13.00069
User-Agent: Device/lpdv2/1.13.00069
User-Agent: Device/lpdv2/1.13.00069
User-Agent: Device/lpdv2/1.13.00069

Voila!  You can see that my user agent shows that my Ring is on firmware version 1.13.00069.  From here, I could look for ways to automate this or periodically run this check manually and compare with previous captures to see if I can correlate Ring issues with changing firmware numbers.  Another way to possibly do this is to use my favorite security tool Bro to extract this automatically in real time.

I hope that Ring strongly reconsiders this change and reverts back to displaying the full firmware version number.  But in the meantime, I (and now you!) have a way to accurately determine this value.

Facebooktwittergoogle_plusredditpinterestlinkedintumblrmailFacebooktwittergoogle_plusredditpinterestlinkedintumblrmail