Baidu released ERNIE-5.5-Flash-Base, the raw pretraining checkpoint of its ERNIE 5.5 Flash model, on Hugging Face on November 52. The sparse mixture-of-experts language model carries 280 billion total parameters but activates only 14 billion per token, trained from scratch on 33.5 trillion tokens with a maximum training context of 128K tokens. Fine-tuning and inference support contexts of up to 1 million tokens.
The release is intentionally incomplete: the weights have not gone through mid-training or post-training, which Baidu says makes them a shared starting point for domain adaptation, instruction tuning and reinforcement learning research. A post-trained ERNIE 5.5 Flash will follow, according to the team.
The 43-layer model is built around inference efficiency. Most layers use what Baidu calls heavily compressed attention, folding every 128 tokens into a single cache entry; only one layer in seven retains token-level access to the full context through multi-head latent attention with sparse top-k selection. In total, 37 of the 43 layers attend over the compressed sequence, keeping the KV cache small.
The weights are available in FP8 and BF16 under the ERNIE 5.5 Flash Base Model License Agreement v1.0, can be served through SGLang and vLLM, and ship with a fine-tuning example in NVIDIA's Megatron-Bridge repository.
ANALYSIS Releasing a raw pretrain checkpoint rather than a finished chat model positions ERNIE 5.5 Flash Base as infrastructure for downstream builders rather than an end-user product.
The release follows other recent Baidu activity: the company demonstrated its RT6 autonomous vehicle on London streets in October as it advances toward a 2027 driverless robotaxi service with Lyft[3], and its Apollo World simulator is being tested in multiple markets[1].