Find the serving recipe that fits the workload.
Balaji Varadarajan from @digitalocean on MOP, a self-learning agent that runs SGLang trials, diagnoses bottlenecks and uses bandit-driven exploration to choose the next experiment.
DigitalOcean's MOP agent runs SGLang trials to find serving recipes
AIDigitalOcean's Balaji Varadarajan describes MOP, a self-learning agent that runs SGLang trials and diagnoses bottlenecks to optimize model serving. MOP uses bandit-driven exploration to choose the next experiment for each workload.
Source details
LMSYS Org · x.comPublished · added here
Original title: Find the serving recipe that fits the workload.