ESP32 Video Streaming in Rust: A Porch Camera That Stays Home

ESP32 video streaming with rusty_esp_video means the camera serves its own picture on your local network: a browser opens an MJPEG page straight off the board, and a laptop or a small hub can take RTP or an MPEG transport stream. On a Seeed XIAO ESP32-S3 Sense it delivered 13.8 frames per second at 320×240 under a 15 fps cap, and ten minutes of unicast RTP lost one packet in 20,120. There is no vendor server in the loop, because nothing in the design needs one.

This field note walks through what the package does, what has actually run on silicon, and what still waits for a board. It is part of the Janus family of memory-safe Rust packages behind the See pillar.

What rusty_esp_video puts on the wire

The package takes frames from an ESP32 camera and turns them into bytes a receiver already understands. It stops at packets: delivery across the wider mesh is the job of rusty_esp_iroh. Everything is pure Rust, with no C and no FFI in the package's own crates, and the core is no_std so it compiles for bare-metal RISC-V and Xtensa targets.

MJPEG over HTTP: the page your browser already reads

The simplest ESP32 video streaming path is the one Espressif's CameraWebServer example made famous: multipart/x-mixed-replace over HTTP, one JPEG per part. The camera already produces JPEG, so the Passthrough encoder hands the sensor's bytes straight to the network without copying them. The frame is borrowed, not duplicated, which matters on a chip with a few hundred kilobytes of fast RAM.

You can run the exact server code on a laptop before touching a board:

cargo run -p rusty_esp_video-esp --features std --example mjpeg_server

That serves colour bars at http://127.0.0.1:8080/stream. On the board the same code runs under ESP-IDF, and the firmware can either join your Wi-Fi or host its own WPA2 network called janus-cam so a camera out of router range still has somewhere to send its picture.

RTP, MPEG-TS and the honest drop policy

For a hub or a recorder, ESP32 video streaming goes beyond the browser page: the package writes RTP/JPEG (RFC 2435), RTP/H.264 (RFC 6184) and H.264 inside an MPEG transport stream. The RTP sender checks that a JPEG uses the standard Annex K Huffman tables before sending it, because the wire carries no tables and every receiver regenerates the standard set. A JPEG with optimised tables is refused rather than shipped as a stream nobody can decode.

The rate control is deliberately blunt. A byte budget, a token bucket in wire bytes, admits a frame only if it fits. A frame that does not fit is dropped whole and counted, never sent late or in pieces. For a doorbell that is the right trade: you want the latest picture of the porch, not a queue of old ones arriving later.

What has run on a real ESP32 camera

Every ESP32 video streaming number below comes from the package's ledger, where no figure appears without the run that produced it. The bench was a XIAO ESP32-S3 Sense hosting its own access point, with a laptop counting from the wire and the board counting its own sends at the same time.

Frame rate, loss, and a cap that was the real limit

• camera page in a browser: result: 500 frames decoded against 503 served, 14.3 fps under a 15 fps cap
• MJPEG, two 108-second arms: result: 13.81 and 13.84 fps, 0.51 Mbit/s
• RTP/JPEG unicast, ten minutes: result: 20,119 packets received, 1 lost (0.005%)
• RTP/JPEG broadcast, ten minutes: result: 2.66% lost

The interesting finding was the video stream's frame rate. The sensor was producing about 27.5 fps and the cap was an integer frame skip, so one in two gives 13.8, not 15. The Wi-Fi link used 0.51 Mbit/s with a 3 ms round trip and had an order of magnitude to spare. The next frames per second will come from a smarter cap, not a better radio.

The gap between unicast and broadcast is the 802.11 standard at work: broadcast frames go out once at the lowest rate with no acknowledgement, while unicast frames are retried. For a home camera, send to an address.

Checked by ffmpeg and rff, both ways

The package's own counters are self-metrics, so every video stream format is also judged by an outside decoder. ffmpeg reads the MJPEG page, and ffprobe reads a transport stream muxed from the house H.264 encoder as h264,64,48,12, every access unit accounted for. The depayloader in turn rebuilds what ffmpeg's own RTP JPEG sender emits, twenty frames with nothing lost or reordered.

The second oracle is rff, the pure-Rust FFmpeg rebuild in remade_ffmpeg_rs. On the host it received MJPEG over HTTP, RTP/JPEG and RTP/H.264 from the Janus senders, and ffprobe counted the result as h264, Constrained Baseline, 320×240, 60 frames. Testing against rff also surfaced a Windows socket-binding bug in rff itself, which was reported and fixed upstream the same day.

A home porch camera: what is ready and what waits

The goal is a porch or doorbell ESP32 camera whose picture goes to your living room, not to a data centre. Pair the video package with rusty_esp_image for capture and rusty_esp_mid for the device's own identity, and the pieces of that camera exist today.

The job picks the codec

A written policy decides the format, so you do not have to. Preview and analytics stay JPEG passthrough. Live streams and archives use H.264 where the chip can afford it: in software on the S3, and on the ESP32-P4's hardware encoder once that backend lands. C-series chips stay on JPEG. Archives are transcoded to AV1 or AV2 later on the home computer, because there is no AV1 on a chip and the cycle budget is not there.

On-chip H.264 is host-verified only today. The encoder compiles for the S3's bare-metal target and its QVGA stream decodes cleanly, but frames per second and PSRAM use on the S3 are still open rows. The P4 hardware encoder is planned, not built, and full WebRTC is out of scope for v1.

Flash it, open it, keep it

The ESP32 camera firmware is an ordinary cargo project. Flashing is covered in the espino guide, and choosing hardware in choosing an ESP32 board. If you have never done this before, deploy an ESP32 at home in an afternoon is the gentle start, and a guided path is coming to dldeploy.

What you get is a camera you can reason about: a page on your LAN, loss counted rather than hidden, and a ledger that says plainly which rows ran on a board. The source lives at github.com/Remade-With-Rust/rusty_esp_video, built with Rust on Espressif's ESP-IDF. For the wider case, read a smart home without the cloud.