Mixture of Experts: Expert Parallelism and the New Inference Stack
Sparse MoE architectures have won the LLM scaling race — here is how to actually run them at production scale.
Distributed-Systems
Sparse MoE architectures have won the LLM scaling race — here is how to actually run them at production scale.