Meta has released Muse Glimmer, a 30-billion parameter multimodal model distributed under the permissive Apache 2.0 license. Distilled from Meta's larger Muse Spark foundation model, Muse Glimmer is engineered specifically for local execution and privacy-sensitive agentic workflows, spanning software engineering, document processing, and desktop automation.
The model release includes immediate day-zero runtime support across Hugging Face Transformers, vLLM, llama.cpp, and native hardware acceleration across NVIDIA CUDA, AMD ROCm, and Intel discrete GPUs and Xeon CPUs.
Dual-Tower Architecture and Hybrid Attention
Muse Glimmer pairs a 2-billion parameter vision tower based on Meta's Perception Encoder with a 28-billion parameter autoregressive text decoder across 52 transformer layers.
The text decoder implements a hybrid attention scheduling mechanism designed to reconcile long-context memory with serving efficiency:
- Interleaved Attention Layers: The network alternates three sliding window attention layers (receptive field of 2,048 tokens using Rotary Position Embeddings) with a fourth layer running full causal attention with No Positional Embeddings (NoPE). This pattern (SWA, SWA, SWA, Full) repeats 13 times across the 52 layers.
- Gated Grouped-Query Attention: Query heads share Key-Value pairs at a 16:1 ratio, cutting KV cache memory overhead by a factor of 16 during inference.
- Q-K Normalization with Query Scaling: Root Mean Square (RMS) normalization is applied to query and key heads prior to attention score computation, stabilized by an explicit post-normalization query scaling factor.
For multimodal processing, the 2B Perception Encoder processes both static images and video clips. Images are patchified into 14x14 tokens with 2D RoPE positional encoding across 50 layers. A 4x spatial pixel shuffle merges neighboring 2x2 token patches prior to decoder projection, preserving channel fidelity while reducing visual sequence length. For video processing, the architecture samples up to 96 frames at 2 frames per second with temporal token placeholders.
Speculative Decoding and Agent Benchmarks
To accelerate code generation and structured outputs, Meta packaged an optional speculative decoding drafter built on DFlash. The drafter runs in parallel with the main decoder to boost output throughput on structured syntax.
On standard agentic evaluations, Muse Glimmer reports competitive marks within the 30B weight class:
- General Agent Tasks: 75.5 on MCP Atlas (outperforming Gemma 4 31B at 54.2 and Qwen 3.6 27B at 62.5), 74.6 on DeepSearch QA, and 43.3 on GAIA 2.
- Coding and Development: 51.2 on SWE-Bench Pro, 76.0 on SWE-Bench Verified, and 51.7 on TerminalBench 2.1.
- Reasoning and Mathematics: 94.7 on AIME 2026 and 83.5 on GPQA Diamond.
- Multimodal Grounding: 78.8 on Charxiv Reasoning and 75.4 on ScreenSpot Pro.
Muse Glimmer weights, inference templates, and deployment scripts are available for direct download on the Hugging Face Hub under Apache 2.0.
