# Mirendil releases Fern, a model pretrained from scratch with 10M-token context

> Original title: Mirendil 发布首个从零预训练的模型 Fern：把 10M token 当成另一种「算力」，服务「做 AI 的人」 @mirendil

- Source: meng shao
- Published: 2026-10-27T03:05:41.000Z
- Bad Influence: https://badinfluence.news/items/cjvzl059t45m3yrxtfwa14vyp
- Original: https://x.com/shao__meng/status/2114916415119770089

## Summary

Mirendil releases Fern 0.1, its first model, pretrained from scratch with up to 10M tokens of context. The company says that is about 23 MB of code, roughly 500,000 lines, or several days of agent work held at once. Fern 0.1 received only a short round of post-training, and Mirendil says next-token prediction keeps improving as context grows to 10M.

## Post

Mirendil 发布首个从零预训练的模型 Fern：把 10M token 当成另一种「算力」，服务「做 AI 的人」 @mirendil 
https://mirendil.com/news/introducing-fern/

10M Token 能做什么：~ 23 MB 代码 / ~50 万行，Megatron-LM 量级，或「几天 agent 干活」的轨迹一次塞进上下文。0.1 版只做了短轮 post-training；Mirendil 叙事是「参数与上下文是两条花算力的路」，架构 / 训练 / serving 都按 10M 设计；next-token 指标随上下文变长仍改善，并称预测质量可一路用到 10M。面向审计数据、读训练日志、通读 agent 轨迹这类高吞吐工作；0.1 只做了短轮 post-training。

六个使用场景
· 更大仓库上的工作
· agentic coding（跑命令、读输出、直到测试过）
· AI 研究代码（训练循环、数据管线、评测）
· 从长日志/堆栈 debug
· 通读整段 agent/训练轨迹并解释因果
· 扫描整块预训练 shard 查重复、榜单污染与垃圾数据

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHVmxQpHbwAA7u9M.jpg&mode=thumb&exp=1793404800&sig=189f0315f7dcbc26)

## Quoted post @bneyshabur

> Introducing Fern, Mirendil’s first model: pretrained from scratch, with up to 10M tokens of context!
> 
> That’s ~0.5M lines of code (roughly Megatron-LM scale) or days of an agent’s work, all in context at once.
> 
> Its first version, Fern 0.1, had a short round of post-training 🚢

https://x.com/bneyshabur/status/2114743262565339148