# Baidu opens ERNIE-5.5-Flash-Base weights as a pre-training base model

> Original title: Baidu is opening ERNIE-5.5-Flash-Base straight out of pre-training: an open-weight foundation for domain adaptation, instruction tuning, ...

- Source: ModelScope
- Published: 2026-11-05T12:59:00.000Z
- Bad Influence: https://badinfluence.news/items/p2j6ii37hp4ru8lug6ganglr4
- Original: https://x.com/ModelScope2022/status/2118327219076673865

## Summary

Baidu has opened the ERNIE-5.5-Flash-Base model weights, a checkpoint taken straight from pre-training before any mid- or post-training. The model has 280B total parameters with 14B activated per token, was trained on 33.5T tokens with a training context up to 128K, and supports up to a 1M-token context for fine-tuning and inference. The post reports MMLU-Pro 78.4, HumanEval+ 82.3, and LiveCodeBench 40.1 under a unified evaluation pipeline, and lists Transformers, SGLang, and vLLM-ERNIE deployment paths.

## Why it matters

The post gives parameter counts, training scale, benchmark scores, and deployment paths, which helps readers judge the base model for their own fine-tuning work.

## Post

Baidu is opening ERNIE-5.5-Flash-Base straight out of pre-training: an open-weight foundation for domain adaptation, instruction tuning, and RL research.
🤖 https://modelscope.ai/models/baidu/ERNIE-5.5-Flash-Base

• 280B total parameters; 14B activated per token
• Pre-trained on 33.5T tokens, with training context up to 128K
• Up to 1M-token context for fine-tuning and inference
• FP8 and BF16 weights

🛠️ A starting point for your own training: The checkpoint precedes all mid- and post-training, leaving developers to choose their own adaptation methods and objectives.

📊 Strong base-model results: Scores 78.4 on MMLU-Pro, 82.3 on HumanEval+, and 40.1 on LiveCodeBench, ahead of the three compared base models on these benchmarks under a unified evaluation pipeline.

🧠 Compressed attention for long inputs: Combines 37 HCA layers, each caching one entry per 128 tokens, with six MLA-DSA layers for full-context access through sparse attention.

⚙️ A built-in speculative draft: ECHO reuses the backbone's final layer to predict future tokens, replacing a separate MTP module.

📦 Deployment: Transformers, SGLang, and vLLM-ERNIE paths are available. The Base model uses the completions endpoint.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWXNlDHa4AA5Ccv.png&mode=thumb&exp=1794096000&sig=b101e46ad506615c)