Tinker releases Dimuon, a LoRA fine-tuning optimizer it says beats AdamW in tests

Tinker (@tinkerapi) introduces Dimuon, a new optimizer it built for LoRA fine-tuning.

The company says that in its supervised fine-tuning tests, Dimuon often reaches lower loss in fewer steps than AdamW. Tinker describes the optimizer as still under experimentation and invites users to try it on Tinker and send feedback.