Skip to content
Trending storyDeveloping

LinkedIn describes serving its ads ranker with SLM and SGLang

2 articles1 sourcesince Nov 4Last article 1h ago ·

Overview

AISummary of 2 articles

LinkedIn engineer Xianxing Zhang explains how the company serves its ads ranker at more than 10K QPS with under 80 ms scoring latency, according to a post shared by the LMSYS Org account on X. The post says the serving stack uses SLM and SGLang.

The reports do not describe the architecture in further detail.

Written by AI from the articles below · updated Nov 4, 5:54 PM ET

Check the sources:

Latest developmentNov 4, 5:51 PM ET
LinkedIn describes serving an ads ranker with SLM and SGLang

Article timeline

The articles in this story. Times are ET.

Nov 4
  1. LMSYS OrgOfficial
    LinkedIn describes serving an ads ranker with SLM and SGLang

    AIXianxing Zhang explains how LinkedIn serves its ads ranker at more than 10K QPS with under 80 ms scoring latency. The post uses SLM and SGLang for the serving stack.

    Image from @lmsysorg's post
  2. LMSYS OrgOfficial
    LinkedIn describes serving an ads ranker with SLM and SGLang

    AIXianxing Zhang explains how LinkedIn serves its ads ranker at more than 10K QPS with under 80 ms scoring latency. The post uses SLM and SGLang for the serving stack.

    Image from @lmsysorg's post

Heat Trend

Not enough continuous observations to show a trend yet.