Inference scales great on GPUs too. On an @nvidia B300, we get over 21k output tokens/s at high concurrency.
3/
Liquid AI reports over 21k output tokens/s on Nvidia B300 GPU
AISummary
Liquid AI says its model reaches more than 21,000 output tokens per second at high concurrency on an Nvidia B300 GPU. The post is the third in a thread, and the source gives no model name or further details.
Post on XView on X
Source: Liquid AI · x.comPublished · added here