mixture-of-experts

Microsoft MAI-1-Preview is an in-house mixture-of-experts foundation model designed for instruction-following and providing helpful responses to everyday queries, currently ranking 13th on LMArena.

Microsoft MAI-1 is a large language foundation model developed by Microsoft AI for instruction-following and conversational AI tasks, trained on approximately 15,000 NVIDIA H100 GPUs.