Together AI introduces Together Compiler for continuous AI inference optimization
AITogether AI introduces Together Compiler (TKCC), a continual optimization loop that uses agents to tune inference for changing traffic and hardware. The company says TKCC reaches up to 2.6x higher TPS across frontier open-source models on SemiAnalysis' InferenceX AgentX benchmark, and 1.7x over open-source megakernels on Llama 1B at batch size 1.
AIWhy it matters.
The post explains how continuous agent-driven recompilation tracks shifting traffic and hardware, with benchmark numbers against megakernels and InferenceX baselines.
Loading source posts…
Source details
Together AI Blog · together.aiPublished · added here
Original title: Together Compiler: Continuous Compilation for AI inference