implementing-ebpf-security-monitoring

What this skill does

# Implementing eBPF Security Monitoring

## When to Use

- When deploying kernel-level runtime security monitoring on Linux hosts or Kubernetes clusters
- When you need sub-millisecond visibility into process execution, network connections, and file access
- When traditional userspace monitoring tools introduce unacceptable performance overhead
- When building detection pipelines that require in-kernel filtering before events reach userspace
- When enforcing runtime security policies (kill process, send signal) at the kernel level

## Prerequisites

- Linux kernel 5.3+ with BTF (BPF Type Format) support enabled
- Kubernetes 1.24+ cluster (for Kubernetes deployment) or standalone Linux host
- Helm 3.x installed (for Kubernetes deployment)
- `kubectl` configured with cluster access
- `tetra` CLI installed for local event streaming
- Python 3.8+ with `requests`, `kubernetes`, `pyyaml` dependencies
- Root or CAP_BPF/CAP_SYS_ADMIN capabilities for eBPF program loading

## Instructions

### 1. Install Tetragon on Kubernetes

Deploy Tetragon via Helm to get default process lifecycle observability:

```bash
helm repo add cilium https://helm.cilium.io
helm repo update
helm install tetragon cilium/tetragon -n kube-system \
  --set tetragon.enableProcessCred=true \
  --set tetragon.enableProcessNs=true
```

Verify the installation:

```bash
kubectl get pods -n kube-system -l app.kubernetes.io/name=tetragon
kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon -c export-stdout -f | head -20
```

### 2. Install Tetragon on Standalone Linux

For non-Kubernetes Linux hosts, install from the tarball release:

```bash
curl -LO https://github.com/cilium/tetragon/releases/latest/download/tetragon-linux-amd64.tar.gz
tar xzf tetragon-linux-amd64.tar.gz
sudo cp tetragon /usr/local/bin/
sudo cp tetra /usr/local/bin/

# Start tetragon daemon
sudo tetragon --btf /sys/kernel/btf/vmlinux &

# Stream events
tetra getevents -o compact
```

### 3. Monitor Process Execution (Default)

Tetragon generates `process_exec` and `process_exit` events by default without any TracingPolicy:

```bash
# Stream process events in compact format
tetra getevents -o compact

# Stream in JSON for SIEM ingestion
tetra getevents -o json | jq '.process_exec // .process_exit'
```

Example `process_exec` JSON event:

```json
{
  "process_exec": {
    "process": {
      "binary": "/usr/bin/curl",
      "arguments": "https://malicious.example.com/payload",
      "cwd": "/tmp",
      "uid": 1000,
      "pod": {
        "namespace": "default",
        "name": "webapp-7b4d9f8c6-x2k9p"
      },
      "parent": {
        "binary": "/bin/bash",
        "pid": 1234
      }
    }
  }
}
```

### 4. Author TracingPolicy for File Access Monitoring

Create a TracingPolicy CRD to monitor access to sensitive files via the `sys_openat` kprobe:

```yaml
# file-access-monitor.yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: monitor-sensitive-file-access
spec:
  kprobes:
    - call: "fd_install"
      syscall: false
      args:
        - index: 0
          type: "int"
        - index: 1
          type: "file"
      selectors:
        - matchArgs:
            - index: 1
              operator: "Prefix"
              values:
                - "/etc/shadow"
                - "/etc/passwd"
                - "/etc/sudoers"
                - "/root/.ssh/"
                - "/etc/kubernetes/pki/"
          matchActions:
            - action: Post
```

Apply and observe:

```bash
kubectl apply -f file-access-monitor.yaml
tetra getevents -o compact --process-filter "event_set:PROCESS_KPROBE"
```

### 5. Author TracingPolicy for Network Connection Monitoring

Monitor outbound TCP connections using the `tcp_connect` kprobe:

```yaml
# network-monitor.yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: monitor-tcp-connections
spec:
  kprobes:
    - call: "tcp_connect"
      syscall: false
      args:
        - index: 0
          type: "sock"
      selectors:
        - matchActions:
            - action: Post
```

### 6. Author TracingPolicy for Privilege Escalation Detection

Detect setuid/setgid calls that may indicate privilege escalation:

```yaml
# privilege-escalation-detect.yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: detect-privilege-escalation
spec:
  kprobes:
    - call: "__sys_setuid"
      syscall: false
      args:
        - index: 0
          type: "int"
      selectors:
        - matchArgs:
            - index: 0
              operator: "Equal"
              values:
                - "0"
          matchActions:
            - action: Post
    - call: "commit_creds"
      syscall: false
      args:
        - index: 0
          type: "cred"
      selectors:
        - matchActions:
            - action: Post
```

### 7. Runtime Enforcement with Sigkill Action

Block unauthorized binary execution by killing the process in-kernel:

```yaml
# enforce-binary-allowlist.yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: enforce-no-crypto-miners
spec:
  kprobes:
    - call: "sys_execve"
      syscall: true
      args:
        - index: 0
          type: "string"
      selectors:
        - matchArgs:
            - index: 0
              operator: "Postfix"
              values:
                - "xmrig"
                - "minerd"
                - "cpuminer"
                - "cryptonight"
          matchActions:
            - action: Sigkill
```

### 8. Export Events to SIEM

Configure Tetragon to export JSON events to a file sink for Fluentd/Filebeat/Vector ingestion:

```bash
# Helm values for file export
helm upgrade tetragon cilium/tetragon -n kube-system \
  --set tetragon.exportFilename=/var/log/tetragon/tetragon.log \
  --set tetragon.exportFileMaxSizeMB=100 \
  --set tetragon.exportFileMaxBackups=5
```

Then configure your log shipper (e.g., Filebeat) to tail `/var/log/tetragon/tetragon.log` and send to your SIEM.

### 9. Kubernetes-Aware Namespace Filtering

Use `TracingPolicyNamespaced` to scope monitoring to specific namespaces:

```yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
  name: monitor-production-file-access
  namespace: production
spec:
  kprobes:
    - call: "fd_install"
      syscall: false
      args:
        - index: 0
          type: "int"
        - index: 1
          type: "file"
      selectors:
        - matchArgs:
            - index: 1
              operator: "Prefix"
              values:
                - "/etc/shadow"
                - "/etc/passwd"
```

## Examples

### Detect Reverse Shell Connections

```yaml
# reverse-shell-detect.yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: detect-reverse-shells
spec:
  kprobes:
    - call: "tcp_connect"
      syscall: false
      args:
        - index: 0
          type: "sock"
      selectors:
        - matchBinaries:
            - operator: "In"
              values:
                - "/bin/bash"
                - "/bin/sh"
                - "/usr/bin/python3"
                - "/usr/bin/perl"
                - "/usr/bin/nc"
                - "/usr/bin/ncat"
          matchActions:
            - action: Post
```

### Monitor Container Escape Attempts

```yaml
# container-escape-detect.yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: detect-container-escape
spec:
  kprobes:
    - call: "sys_openat"
      syscall: true
      args:
        - index: 0
          type: "int"
        - index: 1
          type: "string"
      selectors:
        - matchArgs:
            - index: 1
              operator: "Prefix"
              values:
                - "/proc/1/root"
                - "/proc/1/ns"
                - "/sys/kernel/security"
                - "/proc/sysrq-trigger"
          matchActions:
            - action: Post
    - call: "sys_mount"
      syscall: true
      args:
        - index: 0
          type: "string"
        - index: 1
          type: "string"
        -