# Microsoft and Tsinghua study finds most GPU kernel library failures stem from compilers and environments

> Original title: New Microsoft and Tsinghua paper finds that most GPU kernel library failures in LLM systems come from compilers, builds, and environment ...

- Source: Rohan Paul
- Published: 2026-11-11T05:25:48.000Z
- Bad Influence: https://badinfluence.news/items/topqbpg92pw1l6nzofkacgvj3
- Original: https://x.com/rohanpaul_ai/status/2120387495699587415

## Summary

Microsoft and Tsinghua University analyzed 300 closed GitHub issues from 7 GPU kernel libraries, including FlashAttention, Triton, and CUTLASS. They found 60.33% of failures traced to the compiler, build system, or runtime environment, and 39.67% to the kernel library itself. Software incompatibility among drivers, CUDA, frameworks, and packages was the largest single cause, at 76 issues or 25.33%.

## Post

New Microsoft and Tsinghua paper finds that most GPU kernel library failures in LLM systems come from compilers, builds, and environment mismatches, not kernel code.

LLM training and serving depend on GPU kernel libraries like FlashAttention, Triton, and CUTLASS. When they break, jobs can crash, hang, or silently produce wrong outputs, and the cause is hard to trace.

The team read 300 closed GitHub issue reports from 7 of these libraries, including the discussions and fixes. They labeled each one by what the user saw and what caused it.

Of the 300 issues, 60.33% traced to the compiler and build system or the runtime environment, and 39.67% to the kernel library. The largest single cause was software incompatibility among drivers, CUDA, frameworks, and packages, at 76 issues or 25.33%.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHW0hIK-bkAAghyl.png&mode=thumb&exp=1794700800&sig=20402264fa6f21d7)