cam01 Vision Capture Pipeline

Status: Tier 2 Architecture Document Last Updated: 2026-05-03 WI: WI-395, WI-400

Context

caneast-site1-ot2-cam01 is a Raspberry Pi 5 4GB vision node deployed in zone OT-2 (main floor residential). Its primary purpose is event-driven door-entry detection: monitoring a door via an MC-38 reed switch (GPIO 17) and capturing still images on open events for vision-language model inference to classify what entered or exited.

The pipeline combines hardware sensors, local frame capture, continuous RTSP streaming, Frigate NVR ingestion, and tiered vision inference with local-first sovereignty.

Architecture Overview (Implemented, as of WI-400)

Pi 5 (caneast-site1-ot2-cam01 -- REDACTED)
    |
    +-- MC-38 Reed Switch (GPIO 17, pull_up=True)
    |       |
    |       v
    |   cam01-capture.service (runs as cam01:cam01, video+gpio groups)
    |       |
    |       +-- on_door_open():
    |       |     1. MQTT publish door_state=1, capture_triggered=1
    |       |     2. sudo systemctl stop cam01-mediamtx    [release camera device]
    |       |     3. rpicam-still -> /tmp/cam01_latest.jpg (1920x1080)
    |       |     4. sudo systemctl start cam01-mediamtx   [restore stream, try/finally]
    |       |     5. HTTP POST JPEG to Ollama /api/generate (qwen3-vl:4b)
    |       |     6. MQTT publish inference_result (JSON)
    |       |     7. JSONL append to /var/log/cam01/inference.jsonl
    |       |
    |       +-- on_door_closed():
    |             MQTT publish door_state=0
    |
    +-- cam01-mediamtx.service (runs as mediamtx:mediamtx, SupplementaryGroups=video)
            |
            +-- MediaMTX v1.18.1 (rpiCamera source, softwareH264, 1920x1080, 5fps)
            |
            v
        rtsp://REDACTED:[REDACTED]/cam01
            |
            v
        Frigate 0.14.1 (archon-vision namespace, caneast-site1-node4)
        [Records continuous stream, motion events to MQTT]
            |
            v
        https://frigate.peries.ca  (Traefik IngressRoute, TLS)

Camera contention model (Option A): MediaMTX holds the camera device 100% of the time during normal streaming. On a door-open event, capture.py stops MediaMTX via sudo systemctl, takes a single still with rpicam-still --immediate, then restarts MediaMTX. The try/finally guard ensures the stream is always restored even if capture fails. Stream gap per door event is approximately 2 seconds.

Node Provisioning

OS: Raspberry Pi OS Lite 64-bit Trixie (Debian 13, kernel 6.12 LTS, aarch64). IP: REDACTED (DHCP-reserved). Camera: Camera Module 3 (IMX708, /dev/media2 + /dev/media0 ISP).

Bootstrap pattern follows the fleet-wide two-user model:

Phase	SSH user	How provisioned	Inventory ansible_user
Imager flash	operator	Pi Imager -- operator pubkey loaded from Infisical	operator
Bootstrap play	operator	Same key	operator (transitional)
Ongoing automation	ansible-ot-svc-account	Created by bootstrap play; key from Infisical	ansible-ot-svc-account

Steps to reach automation-ready state: 1. Flash Pi 5 with Raspberry Pi OS Lite 64-bit Trixie via Pi Imager 2. Imager advanced settings: username operator, paste operator SSH public key from Infisical 3. Boot Pi, reserve DHCP IP, set ansible_host: REDACTED in ansible/inventories/ot/hosts.yml 4. Run ansible/playbooks/ot/cam01-bootstrap.yml (creates ansible-ot-svc-account, deploys key, restricted sudoers) 5. Update ansible_user: ansible-ot-svc-account in ot/hosts.yml 6. Run ansible/playbooks/ot/cam01-mediamtx.yml then ansible/playbooks/ot/cam01-capture.yml

Python isolation: Trixie enforces PEP 668 (externally-managed Python). The capture pipeline runs from a virtualenv at /opt/cae/cam01/.venv created with --system-site-packages so that apt-managed packages (gpiozero, paho-mqtt, requests) are accessible. lgpio (python3-lgpio) is system-installed; the service sets WorkingDirectory=/run/cam01 so lgpio can create its notification pipe (Pi 5/Trixie requires writable CWD for lgpio, not HOME).

---

Pipeline Stages

1. Event Source: MC-38 Reed Switch (GPIO 17)

The MC-38 magnetic contact switch is wired to GPIO 17 on the Pi 5 with pull_up=True. gpiozero uses the lgpio backend (python3-lgpio, v0.2.2) to access /dev/gpiochip*.

Event trigger logic: - when_pressed (GPIO pulled low = door open): triggers full capture + inference cycle - when_released (GPIO high = door closed): publishes door_state=0, no capture

2. MQTT Publish (Pre-Capture)

Before capture, publish to MQTT broker at REDACTED:1883: - caneast/ot-zone/cam01/door_state = "1" (retained) - caneast/ot-zone/cam01/capture_triggered = "1"

3. Frame Capture: rpicam-still subprocess

capture.py releases the camera from MediaMTX, then runs:

rpicam-still --immediate --output /tmp/cam01_latest.jpg \
    --width 1920 --height 1080 --nopreview

The --immediate flag skips the AEC/AWB settling delay for fast event capture. After capture (success or failure), MediaMTX is restarted via the try/finally guard.

4. NVR Ingest: MediaMTX RTSP to Frigate

MediaMTX runs as a dedicated systemd service (cam01-mediamtx.service) with the mediamtx system user in the video group. It streams the IMX708 camera at 1920x1080, 5fps, softwareH264 (OpenH264) to Frigate.

See mediamtx-cam01.md for full MediaMTX configuration and the Pi 5 hardware H264 limitation that requires softwareH264.

Frigate role: - Continuous recording with configurable retention - Motion event detection, MQTT publish on events - Web UI at https://frigate.peries.ca and HA integration - Live stream at rtsp://REDACTED:[REDACTED]/cam01, HLS at :8888, WebRTC at :8889

5. VL Inference: Tiered Chain

Tier 1: Ollama (Active, Primary)

• Endpoint: http://REDACTED:[REDACTED]/api/generate
• Model: qwen3-vl:4b
• Latency target: < 90s (timeout)

Tiers 2-4: Cloud Fallbacks (Stub -- not yet implemented)

The inference loop in capture.py has the tier structure defined but Tiers 2-4 are commented stubs. When Tier 1 exhausts, inference_failed is published. Cloud API clients will be added in a future WI when offline resilience is required.

6. MQTT Result Publish and JSONL Logging

{
  "label": "person",
  "confidence": 0.91,
  "description": "A person is visible entering through the doorway",
  "tier_used": 1,
  "model": "qwen3-vl:4b",
  "latency_ms": 4820,
  "ts": "2026-05-03T04:30:00.123Z"
}

Published to caneast/ot-zone/cam01/inference_result (retained). Appended to /var/log/cam01/inference.jsonl with daily logrotate, 90-day retention.

---

Design Decisions

Option A: MediaMTX Owns Camera, rpicam-still for Events

Chosen over: - Option B (picamera2 unified): picamera2 switch-mode is complex; MediaMTX can't use the camera when picamera2 holds it. - Option C (libcamera-vid pipe): brittle coordination via named pipes; failure recovery is complex.

Option A gives MediaMTX full ownership during streaming, and rpicam-still provides clean single-shot captures with --immediate. Stream gap is ~2s per door event, which is acceptable for a 5fps NVR stream.

softwareH264 on Pi 5

The Pi 5 (BCM2712/RP1) does not expose the VC4-style V4L2 M2M H264 encoder that Pi 4 had. MediaMTX's hardwareH264 codec fails with encoder_create(): unable to open device. softwareH264 (OpenH264 via MediaMTX's bundled libcamera v0.5.0+59) works reliably at 5fps on the Pi 5's quad-core Cortex-A76.

lgpio WorkingDirectory

The python3-lgpio library on Pi OS Trixie creates notification pipes relative to the process CWD (not HOME). System users running under systemd have no home directory. The service sets RuntimeDirectory=cam01 (creates /run/cam01 owned by cam01) and WorkingDirectory=/run/cam01 so lgpio can write its .lgd-nfy* pipes.

Ollama First, Cloud Stubs

Per LLMOPS-0002 addendum (2026-05-02): OT-2 is residential/consumer; cloud inference is permitted as fallback. Ollama on the CanEast AI Node workstation (REDACTED:[REDACTED], LAN-bound) is the primary sovereign tier with zero data egress in the normal path.

---

Failure Modes

Mode	Impact	Mitigation
MediaMTX fails to restart after capture	Stream down until service auto-restarts	Restart=on-failure, RestartSec=5 in systemd unit
All inference tiers fail	inference_failed published	Alert on label=inference_failed
Ollama unavailable	No cloud fallback (tiers 2-4 are stubs)	Currently: inference_failed; future WI adds cloud fallback
Camera device busy	rpicam-still fails, exception raised	try/finally still restarts MediaMTX; capture_triggered without inference_result
MQTT broker down	Results not published but JSONL still written locally	paho reconnect on next event

---

References

• OT-0001 (2026-05-02 addendum) -- caneast-site1-ot2-cam01 hardware spec, GPIO map, MQTT topics
• LLMOPS-0002 (2026-05-02 addendum) -- Residential OT-2 zone sovereignty clarification
• platform/mediamtx-cam01.md -- MediaMTX deployment, Pi 5 notes, softwareH264 rationale
• platform/frigate-caneast-site1-node4.md -- Frigate NVR Kubernetes deployment reference
• platform/ollama-alienware-bind.md -- Ollama LAN bind, model inventory
• internal/vl-inference-cam01.md -- qwen3-vl:4b inference details and synthetic test results
• WI-395 -- caneast-site1-ot2-cam01 Phase 1 documentation
• WI-400 -- cam01 MediaMTX integration and Option A camera contention implementation