MoE

An architecture where only a subset of model parameters activate per token, dramatically reducing compute cost. DeepSeek V3 (671B total, 37B active) and Llama 4 Scout (109B total, 17B active) use MoE.

LLM Inference

Build with MoE on XALEN's API.

Get Started