Apple researchers pretrain a tri-modal masked diffusion model from scratch

Apple Machine Learning Research reports the first tri-modal Masked Diffusion Model pretrained from scratch on text, image-text, and audio-text data.

The authors analyze multimodal scaling laws, modality mixing ratios, noise schedules, and batch-size effects, and propose an SDE-based reparameterization that decouples physical from logical batch size.

A preliminary 3B-parameter model trained on 6.4T tokens performs strongly on text generation, text-to-image, and text-to-speech tasks.