Skip to content
View original post on X: Georgi Gerganov· 34/100AI score34/100

llama.cpp adds decision models via new /v1/systemone endpoint

AISummary

Georgi Gerganov announced that decision models are now available in the latest llama.cpp builds through a new /v1/systemone endpoint. The endpoint supports local, private inference with multiple open models, and more are planned.

Post on XView on X
Georgi Gerganov
@ggerganov

Decision models in llama.cpp are now available

The /v1/systemone endpoint is available in the latest llama builds. Use it to do Jev-style inference locally, efficiently and privately. Multiple open models are supported with more to come.

https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp

Source: Georgi Gerganov · x.comPublished · added here

Topics

On-device AIOpen-source ecosystemDeployment & engineeringProduct updates