image_publisher_client — the ROS 2 host side

This directory holds the client half of the camera ingestion stack. The server halves (one per hardware pipeline) live elsewhere — see the parent overview.

The client side is uniform: for every MJPG URL coming off an edge node, run exactly one image_publisher_node in its own tmux pane (or its own Node in a launch file). Each pane:

• decodes the MJPG / JPG stream,
• republishes it as sensor_msgs/Image on <__ns>/image_raw,
• runs at the rate set by publish_rate.

The single most important rule is that publish_rate must equal the edge capture rate exactly. See Why publish_rate matters below.

Files in this directory

A second, ESP32-S3-only Python launch file lives one level up: ../xiao_sense_esp32s3_eyes.py — for the WiFi XIAO eyes (172.31.1.139/142/143/144 @ :81/stream, 16 Hz).

Choosing between image_publishers.tmuxp.yml and pixi_image_publishers.yml

Functionally identical. The only difference is how ros2 is invoked:

# image_publishers.tmuxp.yml — assumes ROS is installed system-wide
- shell_command:
    - ros2 run image_publisher image_publisher_node ...

# pixi_image_publishers.yml — assumes pixi env "kilted" is set up locally
- shell_command:
    - pixi run -e kilted ros2 run image_publisher image_publisher_node ...

Use cases:

• image_publishers.tmuxp.yml — running on the ROS host (the ros2 Docker container, a real Ubuntu/ROS box, or any environment where which ros2 works at the prompt). This is the production launcher when the kernel host comes up.
• pixi_image_publishers.yml — running on macOS / a laptop during development. There is no native ROS 2 install; pixi brings in a per-project ROS distribution (here, the kilted env defined in the repo's pixi config). Use this when you want to view the streams from your laptop using rqt_image_view or Foxglove without spinning up a Docker container.

The pixi variant is intentionally a subset of the full client — it only launches the two cameras you typically need to monitor from a laptop (172.31.1.97:8000 and :8001).

Camera namespace map

The default tmuxp session (image_publishers.tmuxp.yml) publishes the following ROS 2 topics. 172.31.1.x here are the static IPs of the Raspberry Pi hosts running simple_picamera2_streamer/app.py (IMX708 cams over CSI). Each Pi may host more than one streamer on different ports.

All published at publish_rate:=8. — matching the IMX708 streamer's hard 8 Hz cap.

The ESP32-S3 XIAO fleet (xiao_sense_esp32s3_eyes.py):

All published at publish_rate:=16..

The wired-ETH ESP32-S3 + Lepton mix (esp32s3_eth.tmuxp.yml):

IP convention reminder. ESP32-S3 boards on this network are bound to 172.31.1.<DEVICE_ID> by their firmware (#define DEVICE_ID …). Pi cam hosts use whatever DHCP/static you've assigned them; the "port +1 per cam" trick (:8000, :8001) is what lets one Pi host two IMX708 streamers.

Running them

# default — ROS host / docker container
tmuxp load ros2_ws/launch/image_publisher_client/image_publishers.tmuxp.yml

# from a Mac/laptop (no system ROS)
tmuxp load ros2_ws/launch/image_publisher_client/pixi_image_publishers.yml

# wired ESP32-S3 fleet + Lepton thermal
tmuxp load ros2_ws/launch/image_publisher_client/esp32s3_eth.tmuxp.yml

# WiFi XIAO ESP32-S3 fleet
ros2 launch ros2_ws/launch/xiao_sense_esp32s3_eyes.py

Verify in another shell:

ros2 topic list | grep image_raw
ros2 topic hz /cam0/image_raw          # should report ~8 Hz
ros2 run rqt_image_view rqt_image_view /cam0/image_raw

Why publish_rate matters

image_publisher_node opens MJPG URLs through OpenCV (cv::VideoCapture), which buffers decoded frames internally. If publish_rate is slower than the rate the edge produces frames, that buffer fills up and the node republishes frames from seconds — sometimes tens of seconds — in the past.

The downstream symptom is "smooth but wrong" video: rqt_image_view / Foxglove / RViz show motion that looks fluid but lags the real world by a slowly growing offset.

The fix is to match the rate exactly. Per pipeline:

If you bump the edge rate in the firmware / streamer, bump it here in lockstep — every tmuxp file and every launch file in this directory.

Related

• Architectural overview: ros2_ws/edge/README.md
• IMX708 edge server: ros2_ws/edge/simple_picamera2_streamer/README.md
• ESP32-S3 firmware: PhotogrammetricWAAM-Edge/.../CameraWebServer_for_esp-arduino_3.0.x/PROJECT_README.md
• DSLR (role-1 only) pipeline: mqtt__gphoto2_delegate
• xiao_sense_eyes.tmuxp.yml (browser/operator view) at INBOX/TMUXP_VIEWS/