# Santiago explains YaFSDP, an open-source tool for faster multi-GPU LLM training

> Original title: This is an open-source tool for training large language models across multiple GPUs.

- Source: Santiago
- Published: 2026-11-12T16:45:40.000Z
- Bad Influence: https://badinfluence.news/items/n355s17mfnzoyws8jbhqlp1qj
- Original: https://x.com/svpino/status/2120920975940497662

## Summary

Santiago describes YaFSDP, an open-source tool for training large language models across multiple GPUs. It reuses memory and transfers model data while GPUs run other calculations, which speeds up training. Yandex says training Llama 3 70B on 256 A100 GPUs took about 26% less time than with PyTorch FSDP.

## Post

This is an open-source tool for training large language models across multiple GPUs.

It's pretty efficient:

It reuses memory and transfers model data while your GPUs run other calculations, making training much faster.

Yandex published an example where training Llama 3 70B on 256 A100 GPUs took about 26% less time than with a PyTorch FSDP setup.

## Quoted post @yandexcom

> Our technology helps train LLMs up to 26% faster — and it's open source.
> 
> Training a model takes a lot of computing power, but coordinating the work across multiple GPUs can leave up to 30% of their capacity underutilized. GPUs spend time waiting for one another instead of training.
> 
> This is why we developed YaFSDP and subsequently released it as open source.

https://x.com/yandexcom/status/2120537533344104699