How does it work under the hood?
It uses two Gemma models running on Google’s LiteRT runtime:
1️⃣ Conversation & Tool Calling:
When you speak to Reachy Mini by voice. Gemma 4 E4B runs locally to hold natural conversations and issues a tool call to start or stop the car.
2️⃣ Real-Time Vision & Driving:
Once in motion, a small Gemma-based decision model (EmbeddingGemma 2) runs directly on an @arduino VentunoQ embedded board to steer the robot using real-time camera input.
Because both models run on LiteRT, the whole loop stays completely private, offline, and with low-latency.
Check out @IanBallantyne’s breakdown for more technical details on the build!