Mixture-of-Experts
An artificial intelligence design that activates only a small subset of neural networks for each task to save computing power.
This architecture divides a large model into smaller sub-networks called experts. A router component decides which expert handles specific parts of the input data. This allows the total model size to be massive while keeping the computational cost per token relatively low.
