Giving LeKiwi a Voice: Audio Indicators
Implementing spoken status announcements in ROS 2 using Kokoro
Back when I wrote about adding voice control to LeKiwi, I got the reSpeaker Flex mounted, put the mic array through its paces, and ended the post without testing the speaker, which was wired in but silent. I hadn't configured playback yet, which meant I couldn't test anything that needed audio going out - most notably Acoustic Echo Cancellation (AEC), which only makes sense once there's something to echo.
I finally got around to making the speaker work, but not in the way I expected. My original plan was to first test AEC and then jump straight to the fun part - wake word detection, speech-to-text, integration with AI agents such as Claude or nanobot, and finally text-to-speech. Instead, I found a much smaller and more immediately useful problem that I could fix with a speaker.
The problem
If you've read Under the Hood: LeKiwi's Joystick Controls, you'll know that every button-triggered feature on LeKiwi - e-stop, switching between teleop and Nav2, waypoint recording and following, saving the map - is a plain std_srvs/srv/SetBool service, callable from the Steam Deck, Foxglove (using either the Service Call panel or my Button extension), a CLI, or anything else that wants to. That's great for control, but it means there's no feedback: to know whether the e-stop actually engaged, or whether I'm in teleop or autonomous mode, I have to rely on how the robot is behaving.
With a working speaker now sitting on the top deck, the obvious first use for it wasn't a conversation - it was just having the robot tell me what it's doing.
With AKROS, my previous robot, I had LED strips that changed color based on robot state, and it was genuinely useful - one glance told me whether the e-stop was engaged without needing to guess or move the joystick to verify. LeKiwi never got the same treatment, mostly because there's nowhere left to attach LEDs. Every mounting point on the chassis is already spoken for, and I still need to fit a docking and charging solution onto it at some point, which only reduces the available space further. The speaker, on the other hand, is already installed from the reSpeaker work, so I didn't have to change any hardware. This made audio indicators the perfect solution given my constraints.
Decoupling the announcements
The naive way to do this would be to have every feature node call some kind of speak() function directly - twist_switch_node speaks when it flips, waypoint_recorder_node speaks when a waypoint is recorded, and so on. I didn't want that. It's the exact coupling problem I already solved once for control: if the announcement logic lives inside each feature node, every new feature needs its own Text-to-Speech (TTS) wiring, and the node has to know or care that a speaker exists at all. This would become quite messy.
So instead, I wrote indicator_node, a standalone node that doesn't touch any feature node at all. It watches ROS 2's built-in service introspection topics - <service>/_service_event - which every service already publishes regardless of who calls it. That means the announcement fires identically whether the request came from a joystick button, a Foxglove panel, or ros2 service call from a terminal, and adding a new announced service is a config change, not a code change in the feature itself.
Each _service_event stream carries a request event and a response event, so indicator_node matches the two up, remembers what was asked for, and only speaks once the response confirms whether the call actually succeeded. A failed call gets announced too, just with a different phrase - I'd rather hear "No waypoints found" than silence when I try to start a patrol I never recorded.
Centralizing the phrases
I didn't want spoken text scattered across the code either, so every phrase lives in one config file, phrases.yaml, keyed by service name, request value, and outcome:
services:
/emergency_stop:
"true":
success: "Emergency stop enabled"
"false":
success: "Emergency stop disabled"
/twist_switch:
"true":
success: "Autonomous mode"
"false":
success: "Tele-op mode"
/record_waypoint:
"true":
success: "Waypoint recorded"
failure: "Error"
A missing entry means "say nothing," which turned out to matter more than I expected. /record_waypoint, /reset_waypoints, and /save_map are all one-shot SetBool services where data: false does nothing - not a real state to announce - so I just left that entry out of the table entirely.
Pre-rendering with Kokoro
The other decision was when to actually synthesize the audio. Running a TTS model on every announcement, on a Raspberry Pi, in the middle of driving the robot (with computationally heavy features like Nav2, DepthAI), felt like the wrong place to spend CPU - especially since every phrase in phrases.yaml is small, closed-vocabulary, and known ahead of time.
So render_phrases.py (built with Claude's help) runs at build time instead, before setup.py collects data_files, and renders every phrase to a WAV file using Kokoro-82M via onnxruntime. It's incremental too - a phrase that already has a WAV gets skipped, and stale WAVs get pruned, so editing phrases.yaml only re-synthesizes the diff. The 116 MB Kokoro model itself isn't committed to the repo; it gets fetched from Kokoro's GitHub release only when there's actually something new to render, so a clean checkout with unchanged phrases never touches it at all.
What's next
Now that I have a working microphone array, a working speaker, and a working TTS implementation, I can move on to developing the AI-based voice assistant that connects to nanobot or Claude. But first, there are still some things that need to be done. I want to go back and finish what the reSpeaker post left unfinished - testing AEC properly, now that there's real audio coming out of LeKiwi's speaker while the mic is listening.
The second to-do item is to make sure that the indicator node doesn't play a phrase multiple times, as it does now. An example can be seen in the very first video in this post: In my example, the waypoint follower runs infinite patrols, which means that if I register a waypoint on a stationary robot and start the waypoint following, the audio indicator immediately replies with 'goal reached'. Since we have a single waypoint, the patrol loop starts again, and 'goal reached' plays again since the robot hasn't moved. This doesn't stop unless I kill the waypoint follower. This might not be the only issue, and I need to test this feature more extensively.
Beyond that, I honestly don't yet know how I'm going to wire nanobot (or whatever ends up being the AI-powered "brain") into the rest of the ROS 2 stack, let alone how that's supposed to coexist with indicator_node reaching for the same speaker to announce a service call mid-conversation. That's a real problem, and I don't have a good answer for it yet - so I'll leave it for another day.
indicator_node was meant to be a small, self-contained detour, but it ended up proving out the entire playback half of the voice assistant pipeline before I'd written a line of the conversational part. While I keep working on the indicator node investigations and investigating how to implement the pipeline, I've already started planning some hardware upgrades for LeKiwi - a battery monitor and a docking station. More on this when I've made some real progress.
The features described in this post are in the lekiwi_audio package. As always, this package, along with the rest of the LeKiwi ROS 2 stack, lives in lekiwi_ros2; it's a work in progress, so expect breaking or unstable changes once in a while. Meanwhile, feedback and contributions are welcome.
