If you don't run your own LLM proxy, how do you know what's going on?

The author argues that LLM provider dashboards do not give engineers usable token usage data, so they run their own proxy to log which key called which model, from which client, and when.

The proxy also supports virtual API keys, multiple providers and models behind one endpoint, and fast model switching.

The author says they can trace a cache miss in a coding session to its cause within 20 seconds, and that the proxy has an API that could automate this.