Shanghai Jiao Tong's Yan Junchi outlines JTok and xHC Transformer extensions at CNCC 2026
AIShanghai Jiao Tong University professor Yan Junchi presented two Transformer extensions at CNCC 2026: JTok, which adds learnable per-token parameter tables, and xHC, which expands the residual stream to 16 paths with dense reads and sparse writes. In his experiments, JTok-M cuts training compute by about 35% to reach the same loss but adds about 7% inference latency, while xHC saves about 33% versus a standard Transformer and about 16% versus mHC.
Source details
Leiphone · leiphone.comPublished · added here
Original title: Transformer 架构还能咋改？严骏驰团队提供两条思路 | CNCC 2026