performing-network-traffic-analysis-with-zeek

What this skill does

# Performing Network Traffic Analysis with Zeek

## Overview

Zeek (formerly Bro) is an open-source network analysis framework that operates as a passive network security monitor. Unlike traditional signature-based IDS tools, Zeek generates high-fidelity structured logs from observed network traffic, capturing detailed metadata for protocols including HTTP, DNS, TLS, SSH, SMTP, FTP, and dozens more. Zeek's extensible scripting language enables custom detection logic, behavioral analysis, and automated response. This skill covers deploying Zeek, understanding its log architecture, writing custom detection scripts, and integrating outputs with SIEM platforms.


## When to Use

- When conducting security assessments that involve performing network traffic analysis with zeek
- When following incident response procedures for related security events
- When performing scheduled security testing or auditing activities
- When validating security controls through hands-on testing

## Prerequisites

- Linux server (Ubuntu 22.04+ or CentOS 8+) with 4+ CPU cores and 8GB+ RAM
- Network TAP or SPAN port mirroring configured for traffic capture
- Zeek 6.0+ installed (via package manager or source compilation)
- Root or capture group privileges for packet capture
- SIEM platform (Splunk, ELK Stack, or QRadar) for log ingestion

## Core Concepts

### Zeek Architecture

Zeek operates in two main modes:

1. **Live Capture** - Monitors traffic in real-time on one or more network interfaces
2. **Offline Analysis** - Processes saved PCAP files for retrospective analysis

The processing pipeline consists of:

- **Packet Capture Layer** - Reads raw packets from interfaces or PCAP files
- **Event Engine** - Reassembles TCP streams and generates protocol events
- **Script Interpreter** - Executes Zeek scripts that process events and generate logs
- **Log Framework** - Writes structured logs in TSV, JSON, or custom formats

### Log Architecture

Zeek generates protocol-specific log files:

| Log File | Description |
|----------|-------------|
| `conn.log` | TCP/UDP/ICMP connection summaries with duration, bytes, state |
| `dns.log` | DNS queries and responses with query type, answers, TTL |
| `http.log` | HTTP requests/responses with URIs, user agents, MIME types |
| `ssl.log` | TLS handshake details including certificate chain, JA3/JA3S |
| `files.log` | File transfers with MIME types, hashes (MD5, SHA1, SHA256) |
| `notice.log` | Alerts generated by Zeek detection scripts |
| `weird.log` | Protocol anomalies and unexpected behaviors |
| `x509.log` | Certificate details from TLS connections |
| `smtp.log` | Email metadata including sender, recipient, subject |
| `ssh.log` | SSH connection details and authentication results |
| `pe.log` | Portable Executable file metadata |
| `dpd.log` | Dynamic Protocol Detection failures |

## Workflow

### Step 1: Install and Configure Zeek

```bash
# Install Zeek on Ubuntu
sudo apt-get install -y zeek

# Or install from Zeek repository
echo 'deb http://download.opensuse.org/repositories/security:/zeek/xUbuntu_22.04/ /' | \
    sudo tee /etc/apt/sources.list.d/zeek.list
sudo apt-get update && sudo apt-get install -y zeek-lts

# Verify installation
zeek --version
```

Configure the node layout in `/opt/zeek/etc/node.cfg`:

```ini
[manager]
type=manager
host=localhost

[proxy-1]
type=proxy
host=localhost

[worker-1]
type=worker
host=localhost
interface=eth0
lb_method=pf_ring
lb_procs=4

[worker-2]
type=worker
host=localhost
interface=eth1
lb_method=pf_ring
lb_procs=4
```

Configure network definitions in `/opt/zeek/etc/networks.cfg`:

```
# Internal network ranges
10.0.0.0/8         Private RFC1918
172.16.0.0/12      Private RFC1918
192.168.0.0/16     Private RFC1918
```

### Step 2: Configure Logging and Output

Edit `/opt/zeek/share/zeek/site/local.zeek`:

```zeek
# Load standard detection scripts
@load base/protocols/conn
@load base/protocols/dns
@load base/protocols/http
@load base/protocols/ssl
@load base/protocols/ssh
@load base/protocols/smtp
@load base/protocols/ftp

# Load file analysis
@load base/files/hash-all-files
@load base/files/extract-all-files

# Load detection frameworks
@load base/frameworks/notice
@load base/frameworks/intel
@load base/frameworks/files
@load base/frameworks/software

# Load additional protocol analyzers
@load policy/protocols/ssl/validate-certs
@load policy/protocols/ssl/log-hostcerts-only
@load policy/protocols/ssh/detect-bruteforcing
@load policy/protocols/dns/detect-external-names
@load policy/protocols/http/detect-sqli

# Enable JA3 fingerprinting
@load policy/protocols/ssl/ja3

# Enable JSON output for SIEM ingestion
@load policy/tuning/json-logs

redef LogAscii::use_json = T;

# Configure file extraction directory
redef FileExtract::prefix = "/opt/zeek/extracted/";

# Set notice email
redef Notice::mail_dest = "[email protected]";
```

### Step 3: Write Custom Detection Scripts

Create detection scripts for common threats:

**Detect DNS Tunneling** (`/opt/zeek/share/zeek/site/detect-dns-tunnel.zeek`):

```zeek
@load base/protocols/dns

module DNSTunnel;

export {
    redef enum Notice::Type += {
        DNS_Tunnel_Suspected
    };

    # Threshold for suspicious DNS query length
    const query_len_threshold = 50 &redef;

    # Track query counts per host per domain
    global dns_query_counts: table[addr, string] of count &default=0 &create_expire=5min;

    # High query volume threshold
    const query_volume_threshold = 100 &redef;
}

event dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count)
{
    if ( |query| > query_len_threshold )
    {
        local parts = split_string(query, /\./);
        if ( |parts| > 3 )
        {
            local base_domain = cat(parts[|parts|-2], ".", parts[|parts|-1]);
            dns_query_counts[c$id$orig_h, base_domain] += 1;

            if ( dns_query_counts[c$id$orig_h, base_domain] > query_volume_threshold )
            {
                NOTICE([$note=DNS_Tunnel_Suspected,
                        $msg=fmt("Possible DNS tunneling: %s queries to %s with long query names",
                                 c$id$orig_h, base_domain),
                        $conn=c,
                        $identifier=cat(c$id$orig_h, base_domain),
                        $suppress_for=30min]);
            }
        }
    }
}
```

**Detect Beaconing Behavior** (`/opt/zeek/share/zeek/site/detect-beaconing.zeek`):

```zeek
@load base/protocols/conn

module Beaconing;

export {
    redef enum Notice::Type += {
        C2_Beacon_Detected
    };

    # Track connection intervals
    global conn_intervals: table[addr, addr, port] of vector of time &create_expire=1hr;

    const min_connections = 20 &redef;
    const jitter_threshold = 0.15 &redef;
}

event connection_state_remove(c: connection)
{
    if ( c$id$resp_p == 80/tcp || c$id$resp_p == 443/tcp )
    {
        local key = [c$id$orig_h, c$id$resp_h, c$id$resp_p];

        if ( key !in conn_intervals )
            conn_intervals[key] = vector();

        conn_intervals[key] += network_time();

        if ( |conn_intervals[key]| >= min_connections )
        {
            local intervals: vector of interval = vector();
            local i = 1;
            while ( i < |conn_intervals[key]| )
            {
                intervals += conn_intervals[key][i] - conn_intervals[key][i-1];
                i += 1;
            }

            # Calculate mean and standard deviation
            local sum_val = 0.0;
            for ( idx in intervals )
                sum_val += interval_to_double(intervals[idx]);

            local mean_val = sum_val / |intervals|;

            local variance = 0.0;
            for ( idx in intervals )
            {
                local diff = interval_to_double(intervals[idx]) - mean_val;
                variance += diff * diff;
            }
            variance = variance / |intervals|;
            local stddev = sqrt(variance);

            if ( mean_val > 0 && (stddev /