@YottaLabs chief scientist Dong Li shares the team’s work on SGLang-Trainium: adding support for new MoE models and optimizing their inference latency on Trainium.
YottaLabs shares SGLang-Trainium work on new MoE models and inference latency
AIYottaLabs chief scientist Dong Li shares the team's work on SGLang-Trainium, adding support for new MoE models. The work also optimizes inference latency for those models on Trainium.
Source details
SGLang · x.comPublished · added here
Original title: @YottaLabs chief scientist Dong Li shares the team’s work on SGLang-Trainium: adding support for new MoE models and optimizing their infe...