Zeekurity Zen – Part II: Zeek Package Manager

Zeekurity Zen – Part II: Zeek Package Manager

If you're looking for professional services on this topic or interested in other cybersecurity consulting services, please reach out to me via my Contact page to discuss further.

This is part of the Zeekurity Zen Zeries on building a Zeek (formerly Bro) network sensor.

Overview

In our Zeek journey thus far, we’ve:

Now we’ll introduce the Zeek Package Manager to extend Zeek’s functionality with packages contributed by the Zeek community.  A full list of available packages can be viewed on the Zeek Package Browser.  We will focus on configuring Zeek to use AF_PACKET to further optimize packet capture and analysis.  We’ll also install additional useful packages.

To do this, we’ll walkthrough these steps:

  1. Set up Zeek Package Manager to extend Zeek’s functionality.
  2. [Optional] Use Zeek Package Manager to install the AF_PACKET package.
  3. Configure Zeek to use the AF_PACKET package to optimize Zeek’s packet capture performance.
  4. [Optional] Install additional useful packages including ja3 and HASSH.
  5. Update Zeek packages.

Set up Zeek Package Manager

  1. As the zeek user, make sure you’re in its respective home directory.
    cd
  2. As the zeek user, install zkg’s dependencies. This will install two external Python modules that zkg requires to ~/.local/lib/python3.6/site-packages.
    pip3 install GitPython semantic-version --user
  3. As the zeek user, configure Zeek Package Manager (zkg).
    zkg autoconfig

    This will create a configuration file in /opt/zeek/etc/zkg/config. Upon completion it should look something like the following.

    zeek = https://github.com/zeek/packages
[paths]
state_dir = /opt/zeek/var/lib/zkg
script_dir = /opt/zeek/share/zeek/site
plugin_dir = /opt/zeek/lib64/zeek/plugins
zeek_dist = /home/zeek/zeek-5.2.1
  4. If zkg is not installed or executed properly, you may see the following error:
    zkg
error: zkg failed to import one or more dependencies:
* GitPython:        https://pypi.org/project/GitPython
* semantic-version: https://pypi.org/project/semantic-version
If you use 'pip', they can be installed like:
     pip3 install GitPython semantic-version
  5. This is most likely due to one of the following reasons:
    • You ran zkg prior to installing the required python modules via pip3.
    • You ran zkg from a directory in which the zeek user does not have write permissions to.
    • You do not have the git package installed.

    Take the necessary steps to resolve the issue and try running it again.

[Optional] Install the AF_PACKET package

Note: As of Zeek version 5.2, AF_PACKET support is now included as part of the base Zeek installation. The following steps for installing the AF_PACKET package are now considered optional unless using a version of Zeek lower than 5.2.

  1. As root/sudo, run the following apt command to install kernel development files.
    sudo apt-get install linux-headers-$(uname -r)
  2. Switch back to the zeek user and stop Zeek if it is currently running.
    zeekctl stop
  3. Use zkg to install the AF_PACKET package.
    zkg install zeek/zeek/zeek-af_packet-plugin
The following packages will be INSTALLED:
  zeek/zeek/zeek-af_packet-plugin (4.0.0)
Proceed? [Y/n] y
Running unit tests for "zeek/zeek/zeek-af_packet-plugin"
Installing "zeek/zeek/zeek-af_packet-plugin".......
Installed "zeek/zeek/zeek-af_packet-plugin" (4.0.0)

Configure Zeek to use AF_PACKET

  1. Edit /opt/zeek/etc/node.cfg to configure Zeek to use AF_PACKET.  In the example configuration below we are configuring one worker, load balanced across two cores, analyzing one sniffing interface.
    # Example ZeekControl node configuration.
# Below is an example clustered configuration on a single host.
[logger]
type=logger
host=localhost
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[worker-1]
type=worker
host=localhost
interface=af_packet::enp2s0
lb_method=custom
lb_procs=2
pin_cpus=0,1

    In the event you have two or more sniffing interfaces (e.g. enp2s0 and enp3s0), see the example configuration below which assigns each interface its own worker, load balanced across two cores, again using AF_PACKET. Note the addition of unique af_packet_fanout_id values for each of the sniffing interfaces.

    # Example ZeekControl node configuration.
# Below is an example clustered configuration on a single host.
[logger]
type=logger
host=localhost
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[worker-1]
type=worker
host=localhost
interface=af_packet::enp2s0
lb_method=custom
lb_procs=2
pin_cpus=0,1
af_packet_fanout_id=2
[worker-2]
type=worker
host=localhost
interface=af_packet::enp3s0
lb_method=custom
lb_procs=2
pin_cpus=2,3
af_packet_fanout_id=3
  2. As root/sudo, give the Zeek binaries permissions to capture packets. This was previously done in Part I, however, installing AF_PACKET requires doing this again.
    sudo setcap cap_net_raw=eip /opt/zeek/bin/zeek
sudo setcap cap_net_raw=eip /opt/zeek/bin/capstats
  3. As the zeek user, run zeekctl deploy to apply configurations and run Zeek.
    zeekctl deploy

    If you see the following errors, try re-running the sudo setcap commands from the previous step.

    zeekctl deploy
Error: worker-1-1 terminated immediately after starting; check output with "diag"
Error: worker-1-2 terminated immediately after starting; check output with "diag"
Error: worker-2-1 terminated immediately after starting; check output with "diag"
Error: worker-2-2 terminated immediately after starting; check output with "diag"

[Optional] Install Additional Useful Packages (e.g. add-interfaces, ja3, and HASSH)

We’ll install additional Zeek packages: add-interfaces, ja3, and HASSH. The install process outlined below should work for installing other packages you may be interested in.

  1. As the zeek user, stop Zeek if it is currently running.
    zeekctl stop
  2. Use zkg to install the add-interfaces package. In situations where you are monitoring multiple network interfaces on one sensor, this adds an “_interface” field to every log file which labels the particular network interface that traffic is coming from.
    zkg install zeek/j-gras/add-interfaces
The following packages will be INSTALLED:
  zeek/j-gras/add-interfaces (master)
Proceed? [Y/n] y
Installed "zeek/j-gras/add-interfaces" (master)
Loaded "zeek/j-gras/add-interfaces"
  3. Edit /opt/zeek/share/zeek/site/add-interfaces/add-interfaces.bro and modify the const enable_all_logs and const include_logs: set[Log::ID] fields as shown below. Save the file when you’re finished.
    export {
        ## Enables interfaces for all active streams
        const enable_all_logs = T &redef;
        ## Streams not to add interfaces for
        const exclude_logs: set[Log::ID] = { } &redef;
        ## Streams to add interfaces for
        const include_logs: set[Log::ID] = { } &redef;
}
  4. Use zkg to install the ja3 package. This is used for profiling SSL/TLS clients.
    zkg install zeek/salesforce/ja3
The following packages will be INSTALLED:
  zeek/salesforce/ja3 (master)
Proceed? [Y/n] y
Installing "zeek/salesforce/ja3"
Installed "zeek/salesforce/ja3" (master)
Loaded "zeek/salesforce/ja3"
  5. Use zkg to install the HASSH package. This is used for profiling SSH clients and servers.
    zkg install zeek/salesforce/hassh
The following packages will be INSTALLED:
  zeek/salesforce/hassh (master)
Proceed? [Y/n] y
Installing "zeek/salesforce/hassh"
Installed "zeek/salesforce/hassh" (master)
Loaded "zeek/salesforce/hassh"
  6. Edit /opt/zeek/share/zeek/site/local.zeek and add the following lines to the bottom. This will load all packages you’ve installed.
    # Load Zeek Packages
@load packages
  7. As the zeek user, run zeekctl deploy to apply configurations and run Zeek.
    zeekctl deploy

Update Installed Zeek Packages

  1. As the zeek user, stop Zeek if it is currently running.
    zeekctl stop
  2. Use zkg to check for updated packages.
    zkg refresh
Refresh package source: zeek
        No changes
Refresh installed packages
        New outdated packages:
                zeek/salesforce/hassh (master)

    This indicates that zeek/salesforce/hassh needs to be updated.

  3. Use zkg to check for updated packages.
    zkg upgrade
The following packages will be UPGRADED:
  zeek/salesforce/hassh (master)
Proceed? [Y/n] y
Upgraded "zeek/salesforce/hassh" (master)
  4. As the zeek user, run zeekctl deploy to apply configurations and run Zeek.
    zeekctl deploy

Up Next

In Part III of this series, we will walkthrough how to send Zeek logs to Splunk and take advantage of the Corelight For Splunk app.

If you're looking for professional services on this topic or interested in other cybersecurity consulting services, please reach out to me via my Contact page to discuss further.

Related Posts

Author:
Filed Under: how-to, incident response, information security, network security monitoring
Tags: ,

25 thoughts on “Zeekurity Zen – Part II: Zeek Package Manager”

  • Hello Eric, Thank you for the guide. I’m having issues with the AF_Packet. It seems I misconfigured it wrong previously with a server with 1 CPU and 4 Cores.
    Before i had a separate worker for each core on the same server. Now i somehow got it to work with the af_packet; previously i kept getting an error that worker got terminated. Somehow i got it to work.

    [worker-1]
    type=worker
    host=localhost
    interface=af_packet::ens37
    lb_method=custom
    lb_procs=4
    pin_cpus=0,1,2,3

    Now when i see status, i can see worker 1-1, 1-2 1-3 1-4.

    So it seems like it’s load balancing? However, the logs just don’t seem right. I’m seeing duplicate entries in conn.log dns.log and intel.log.

    For example, if i set the intel.log like in your guide, and i have a computer connect to it, instead of seeing 1 entry for that connection, i’m seeing 4 entries.
    you can see each entry 4 times, and the only difference isa different timestamp, connection uid, and seen”node.

    i would see the same entry 4 times and you can see worker-1-1, worker 1-2, worker 1-3, worker 1-4 for the same entry. It’s as if that 1 CPU with 4 Cores; each of the core is processing 100% of the logs, instead of that 1 CPU distributing the work across the 4 Cores.

    Is there something I’m doing wrong with AF_Packet? Some googling mentioned this online and said to use PF_RING to load balance across the cores instead? But i think i saw your posts and comments on af_packet being better, but I’m not understanding why i’m getting the duplicate log entries. It makes it difficult to search the logs and analyze behavior of a host.

    any ideas? Thanks

    Reply

    • Hm, that looks good assuming that’s the only interface you’re monitoring. I’d double check the packets you’re sending to the interface. Run a tcpdump without zeek on that ens37 interface and see if you’re seeing duplicate packets. A simple thing to try is running a DNS query and seeing if you see it multiple times. If you do, then your network mirror / network tap is not configured in the way you want.

      Reply

  • Hello, I keep getting this error at Step 3 and followed your instructions. Any ideas?
    I made sure to switch to zeek user and ran these commands before running zkg like stated, but still got the error you mentioned. Tried re-doing it over again and installed as my other user account with sudo privileges and i’m still getting this error. I think i configured this before and got it working, not sure what changed.

    cd
    pip3 install GitPython semantic-version –user

    —
    zkg
    error: zkg failed to import one or more dependencies:

    * GitPython: https://pypi.org/project/GitPython
    * semantic-version: https://pypi.org/project/semantic-version

    If you use ‘pip’, they can be installed like:

    pip3 install GitPython semantic-version

    Reply

    • Did you have a previous version of zeek/zkg installed prior to this? If you did, you can type “which zkg” to see which “zkg” is actually being called. Assuming you installed zeek to /opt/zeek, the output should be “/opt/zeek/bin/zkg.” If it’s not that, then you’ve got another version of zkg installed that needs to first be removed before running through the steps in the guide. You can try “pip3 uninstall zkg –user” to uninstall previous versions that were installed via pip.

      Don’t install via sudo, this will prevent the zeek user from updating it going forward. Uninstall any sudo-installed packages before trying again. It’s generally not recommended to run pip as sudo for security and management reasons. So long as you’re installing as the zeek user and running zkg from a directory it has write permissions to, this should work.

      Reply

      • Still not working. Not sure why. I believe i did this before and it worked and I tried multiple times and followed instructions and it’s not working this time for some reason..
        Ran which zkg and it shows /opt/zeek/bin/zkg

        Still getting same error.

        i installed as zeek user without sudo like i did previously.

        tried running pip3 uninstall zkg -user and it doesn’t work. it just says

        Uage:
        pip uninstall [options] …
        pip uninstall [options] -r ..
        no such option: -u

        No idea why this simple step of zkg is not working.

        If i run pip3 install GitPython semantic-version –user
        it says
        Requirement already satisfied

        Reply

        • Hard to say what it is without seeing it myself. That error only occurs if you’re: 1. Not running it as the zeek user (assuming you’ve run the pip install as zeek) or 2. Not running it from a directory the zeek user has write permissions to. Sounds like something is off, since you’ve done it before, just not sure what. I accept Venmo if you still can’t get it going. 🙂

          Reply

  • Hello Eric,

    When i edit node.cfg to use: (replacing enp2s0 with ens33)

    [worker-1]
    type=worker
    host=localhost
    interface=af_packet::enp2s0
    lb_method=custom
    lb_procs=2
    pin_cpus=0,1

    and run zeekctl deploy, I get this error:
    Error: worker-1-1 terminated immediately after starting; check output with “diag”
    Error: worker-1-2 terminated immediately after starting; check output with “diag”

    I tried running this command as root as suggested but still fails:
    sudo setcap cap_net_raw=eip /opt/zeek/bin/zeek
    sudo setcap cap_net_raw=eip /opt/zeek/bin/capstats

    Any thoughts?
    Thank you for this guide. Hopefully this helps with what I need to do for my job. We are in need of analyzing DNS further and have wireshark captures showing our dns server sending dns query to an external ip address for some odd reason once in awhile and source is coming from our core switch. Other than that, we don’t know what else to gain from this and how to analyze it. Hoping zeek can do something to help wit hthis.

    Reply

    • Hi Kevin,

      Were you able to get Zeek running prior to trying to use AF_PACKET? In other words, was it running successfully after you completed Part I of this guide? It sounds strange, but try running those two sudo commands a couple times. For whatever reason, I’ve seen it not take initially, and then it will go. If you’re on the Zeek Slack, feel free to message me there, too.

      Appreciate the kind words on the guide! Zeek is awesome for DNS analysis. Hopefully we can figure this issue out and get you going.

      – Eric

      Reply

      • Hi Eric, I’m starting over to install Zeek again now that I have more time on this project. I was wondering if you could assist me in answering some questions and also help with the node.cfg config since I’m a bit confused on how that works. I was able to get zeek running prior to this step.

        In Part 1 you stated that it is recommended to use a max of 1 or 2 less workers than the total number of CPU Cores. I’m using an HP EliteDesk 800 G4 SFF Desktop I found with 1 CPU (6 Cores) so would I have 4 Workers configured like this? I plan on using 2 interfaces, 1 sniffing interface, and 1 just to manage the desktop.

        [manager]
        type=manager
        host=localhost
        #
        [proxy-1]
        type=proxy
        host=localhost
        #
        [worker-1]
        type=worker
        host=localhost
        interface=eno1
        #
        [worker-2]
        type=worker
        host=localhost
        interface=eno1
        #
        [worker-3]
        type=worker
        host=localhost
        interface=eno1
        #
        [worker-4]
        type=worker
        host=localhost
        interface=eno1

        If the answer is no, how should i configure this?
        If the answer is yes, then how would I configure the AF_Packet section? Would I configure the parameters (interface=af_packet::eno1, lb_method, lb_procs,pincpu) for each worker or just worker 1?
        Like this?

        [manager]
        type=manager
        host=localhost
        #
        [proxy-1]
        type=proxy
        host=localhost
        #
        [worker-1]
        type=worker
        host=localhost
        interface=af_packet::eno1
        lb_method=custom
        lb_procs=4
        pin_cpus=0,1,2,3
        #
        [worker-2]
        type=worker
        host=localhost
        interface=eno1
        #
        [worker-3]
        type=worker
        host=localhost
        interface=eno1
        #
        [worker-4]
        type=worker
        host=localhost
        interface=eno1

        Thanks!

        Reply

        • Yep, the first configuration will work if you’re not using AF_PACKET.

          When using AF_PACKET, you’ll want to configure those settings for each worker. So those same settings you listed for worker-1, apply those to each subsequent worker and you should be good.

          Reply

          • Thanks I’ll try that out. What hardware would you usually use for Zeek setup? If we are wanting to monitor all the traffic that goes in and out of our Core Switch to Firewall to Internet, would a standard desktop with 1 sniffing interface work?
            HP Elitedesk 800 G5 for example or old server hardware with 2 NICs? I guess you might say it depends on our traffic? Our firewall doesn’t seem to show much bandwidth usage as most users are on vpn nowadays. Not sure if i’m reading it right but average for the last week is around 52 Mbps which doesn’t seem like a lot?

            Just wondering since I’m not sure what I’m doing would be good if i got zeek working and it was on a spare desktop we found or if I need to do this on a higher-end server.
            Thanks!

            Or is it recommended to use a faster server for this?

            Reply

            • Absolutely depends on the traffic, but more specifically, the amount of traffic. 52 Mbps doesn’t seem like much at all, I’d double check that but it could be normal depending on the number of users you normally have and how your VPN is setup (full vs split). Assuming that is accurate, your current hardware will work fine.

              If you’re starting from scratch, I’d use the faster server since the more CPU / RAM / disk you can get your hands on, the higher you can scale over the long run (monitor higher bandwidth, more networks, longer log retention). As you build Zeek out, you’ll likely find that you want to monitor other network segments or build other tools onto that server and having better hardware will make that much easier to do. For an enterprise setup, I’d shoot for at least 8 cores, 24 GB RAM, and 500 GB storage. Again, it depends on your requirements.

              Also, if you need professional help, I do contract work, too. 🙂

              Reply

  • I config the the node.cfg like in your articles. but i face a problem issue ” checking configurations …
    installing …
    removing old policies in /opt/zeek/spool/installed-scripts-do-not-touch/site …
    Error: failed to remove directory /opt/zeek/spool/installed-scripts-do-not-touch/site: [Errno 13] Permission denied: ‘local.zeek’
    When I run zeekctl deploy. How can i resolve this issue?
    Thanks and regards,

    Reply

  • I am new to zeek, and trying to test zeek, I did the first 2 tutorial. Is there a way to test that zeek has started monitoring my network. what should I configure to send my data and check that zeek captured it correctly.
    Also, how to setup zeek to inflow and outflow of network from system to internet?

    Reply

    • Hi Abhishek,

      1. My guide doesn’t cover it (and maybe it should), but you’ll need a way to send network traffic to your Zeek system. This is typically done via a network TAP or mirror/SPAN port that sends a copy of network traffic to your Zeek sensor. Once you’ve got that working, if your Zeek is installed and running successfully, you should be seeing logs generated in /opt/zeek/logs/current.
      2. I’m not sure I understand your last question.

      Hope that helps!

      Reply

  • Hi Eric,
    in step 4 INSTALL ZEEK PACKAGE MANAGER
    it can’t find the file zkg autoconfig after executing pip3 install –user zeek zkg
    I’m using the zeek user, and my system is a RHEL server. Am I missing something?

    Reply

      • Hi Parson,

        After you run the pip3 install command it should have installed zkg to ~/.local/bin/zkg. From there you run zkg autoconfig to automatically create and populate the zkg configuration file in /home/zeek/.zkg/config. The tricky part is that you need to install python3-pip as a user with sudo privileges and then switch users to the zeek user and install zkg.

        Hope that helps!

        Reply

  • Hi Eric, nice posts! very detailed thanks! I was wondering if you ever had this errors before while setting up zkg?
    ~]# zkg autoconfig
    warning: skipped using package source named “zeek”: failed to clone git repo
    Successfully wrote config file to /root/.zkg/config

    ~]# more /root/.zkg/config
    [sources]
    zeek = https://github.com/zeek/packages

    [paths]
    state_dir = /root/.zkg
    script_dir = /opt/zeek/share/zeek/site
    plugin_dir = /opt/zeek/lib/zeek/plugins
    zeek_dist =

    whatever package I try to install, it gives me this error:
    ~]# zkg install zeek/j-gras/bro-af_packet-plugin
    warning: skipped using package source named “zeek”: failed to clone git repo
    error: invalid package “zeek/j-gras/bro-af_packet-plugin”: package name not found in sources and also not a usable git URL (invalid or inaccessible, use -vvv for details)

    Any thoughts?
    many thanks!

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.