Brick · 03

Hear

Microphones, audio pipelines, and DSP on the chip

Why this band exists

Sound tells you more about a home than a camera does, which is exactly why it should never leave without asking. Hear is the audio half of the Janus family: rusty_esp_audio captures from PDM and I2S microphones, encodes PCM, IMA ADPCM, WAV, or FLAC, and streams to your own speakers, while rusty_esp_dsp keeps the math fast and byte-identical to a reference.

Why now

The audio path has run a ten-minute soak on a XIAO ESP32-S3 Sense with zero datagrams lost, and its ADPCM is byte-identical to ffmpeg in both directions. Speaker loopback is still unmeasured, and the plan says so.

How it fits

Flash composes the mic into the image. Connect moves the audio over the mesh. See shares the board for a porch cam with sound. Own keeps the stream subscribed only to the people you adopted.

Bricks in Hear

Each brick is a package in the Janus family under Remade-With-Rust. Read the field note, then snap it onto the rest of the house.

  • rusty_esp_audio · replaces ESP-ADF / esp_codec_dev

    Pipeline, PDM and I2S capture, PCM, ADPCM, WAV, FLAC

    Repo↗Field note

  • rusty_esp_dsp · replaces ESP-DSP C kernels

    Scalar kernel home with a PIE SIMD seam for S3 and P4

    Repo↗Field note

What to build first

  1. A baby monitor that plays on a laptop with ffplay and nowhere else
  2. A room-to-room intercom over the home mesh
  3. A FLAC recorder for a workshop or a practice room
Flagship field noteBrowse the family ↗All Hear notes on Learn→

Field notes in Hear

The rest of the home mesh

Six technology bands, one sky. Open another star, or return to the full map.

Open the Vision map→