Meta Unveils Muse Spark Open, 816-Billion-Parameter Multimodal Model

Meta has unveiled Muse Spark Open, an 816-billion-parameter multimodal model that uses a mixture-of-experts architecture to activate only 42 billion parameters per token, supports a one-million-token context window, and can ingest text, images, audio, and video in a single prompt.