StepFun has released Step 5 Preview, a sparse Mixture-of-Experts model carrying roughly 600 billion total parameters but activating only about 27 billion per token1,2. API access is live now; the company says model weights are scheduled to open on October 15.
The model ships with a one-million-token context window, multimodal support for text and image inputs, and listed API prices of about $1 per million input tokens and $2.7 per million output tokens, with roughly 100 tokens per second output throughput.
Architecture built for depth
Step 5 Preview uses a 92-layer narrow-deep Transformer layout rather than a wider network. StepFun argues that deeper stacks provide longer information paths for implicit multi-hop reasoning during long prefill, the phase when agents search, execute code, and ingest tool returns.
To make million-token sessions practical at that depth, the model adds Sparse Grouped-Query Attention with block-wise token merging. StepFun says the technique cuts indexer and top-k selection cost to about one-eighth of a denser baseline while merging overlapping neighboring selections.
Training and inference stack
Training emphasizes on-policy long-horizon reinforcement learning with bit-wise train–inference alignment across MoE routing. The stack includes load-aware scheduling, MTP-3 speculative decoding, FP8 MoE, and KV-cache offload. StepFun reports more than threefold end-to-end acceleration for long-horizon RL and sample-ledger loss below one percent.
The model is also used inside a human-governed data pipeline that produces million-scale verifiable hard tasks spanning science, software engineering, and machine-learning R&D.
Positioning and target workloads
StepFun targets long-horizon agent workloads including AI coding, software engineering, financial analysis, and professional knowledge work. On Artificial Analysis, Step 5 Preview scores about 44 on the intelligence index; StepFun positions it among the top open-weight models in that ranking.
The release is framed around architecture and agent efficiency, distinct from StepFun's earlier Step 3.5 Flash and Step 3.7 Flash releases, centering instead on depth, sparse long context, and durable multi-step tool use ahead of the October weight drop.
ANALYSIS The 27-billion activated parameter count at 600 billion total places Step 5 Preview in a sparsity regime where per-token compute cost stays comparable to much smaller dense models, while the 92-layer depth and million-token window are tuned specifically for the extended reasoning chains that agentic pipelines demand. Pricing at roughly $1/$2.7 per million input/output tokens, combined with the planned October 15 weight release, puts the model in direct competition with other open-weight flagships on both cost and accessibility.