Project by @mmastrac! He also open-sourced the approach as ‘hunch’ if you want to try it yourself:
Demo post: https://x.com/mmastrac/status/2115098609356284223
GitHub:
https://github.com/undrgrnd-labs/hunch
Gemma-based real-time inference project hunch is open-sourced by mmastrac
AImmastrac has open-sourced hunch, a project that uses EmbeddingGemma2 as a real-time /v1/systemone endpoint. The endpoint runs in about 20ms on text and 100ms on images, is trainable in real time, and can distill an upstream teacher model.
@googlegemma
Part of a thread · earlier post
It's been a while since I showed any Jev stuff but I've been cooking on a way to use @GoogleDeepMind's EmbeddingGemma2 as an insanely fast real-time /v1/systemone endpoint (20ms on text, 100ms on images). It's trainable in real-time, can distill an upstream teacher model, and is multi-modal by default. Hard to believe a @googlegemma model of this size can do so well with the right harness!
Source details
Google Gemma · x.comPublished · added here
Original title: Project by @mmastrac! He also open-sourced the approach as ‘hunch’ if you want to try it yourself: