Baidu is opening ERNIE-5.5-Flash-Base straight out of pre-training: an open-weight foundation for domain adaptation, instruction tuning, and RL research.
🤖 https://modelscope.ai/models/baidu/ERNIE-5.5-Flash-Base
- 280B total parameters; 14B activated per token
- Pre-trained on 33.5T tokens, with training context up to 128K
- Up to 1M-token context for fine-tuning and inference
- FP8 and BF16 weights
🛠️ A starting point for your own training: The checkpoint precedes all mid- and post-training, leaving developers to choose their own adaptation methods and objectives.
📊 Strong base-model results: Scores 78.4 on MMLU-Pro, 82.3 on HumanEval+, and 40.1 on LiveCodeBench, ahead of the three compared base models on these benchmarks under a unified evaluation pipeline.
🧠 Compressed attention for long inputs: Combines 37 HCA layers, each caching one entry per 128 tokens, with six MLA-DSA layers for full-context access through sparse attention.
⚙️ A built-in speculative draft: ECHO reuses the backbone's final layer to predict future tokens, replacing a separate MTP module.
📦 Deployment: Transformers, SGLang, and vLLM-ERNIE paths are available. The Base model uses the completions endpoint.