Apple ML Research releases RSI-Inference Bench for AI agents on inference engineering

Apple ML Research introduces RSI-Inference Bench, a benchmark of real vLLM and SGLang inference engineering tasks for testing whether AI agents can improve the systems that run AI models.

The tasks cover day-0 model support, inference optimization and bug fixes, with binary and continuous verifiable rewards.

Apple ML Research says the best agent currently passes 26.7% of tasks at pass@1, and describes the benchmark as unsaturated. The announcement was made in an X post, and the figure is the source's own result.