Skip to content
Trending storyDeveloping

Tri Dao compares TPU and GPU bottlenecks for kernel design

1 article1 sourcesince Oct 14Last article 2h ago ·

Overview

AISummary of 1 article

Tri Dao, the researcher behind FlashAttention, says TPUs have few cores, so parallelism matters less for TPU kernels than for GPU kernels.

He says TPUs have large VMEM, which makes kernel fusion easier. He also says TPU vector units have lower throughput, so non-matmul operations need more careful handling. Dao made these points in a post on X dated October 14, 2026.

Written by AI from the articles below · updated Oct 14, 6:58 PM ET

Check the sources:

Latest developmentOct 14, 6:43 PM ET
Tri Dao compares TPU and GPU bottlenecks for kernel design

Article timeline

The articles in this story. Times are ET.

Oct 14
  1. Tri DaoX
    Tri Dao compares TPU and GPU bottlenecks for kernel design

    AITri Dao says TPUs have few cores, so parallelism matters less, and large VMEM makes kernel fusion easier. He adds that TPU vector units have lower throughput, so non-matmul operations need much more care.

Heat trend

Current comparable heat
9
Peak comparable heat
10Oct 14, 7:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.