{"id":348310,"date":"2025-06-23T10:34:28","date_gmt":"2025-06-23T17:34:28","guid":{"rendered":"https:\/\/msftnewsnow.com\/?p=348310"},"modified":"2025-06-27T18:58:11","modified_gmt":"2025-06-28T01:58:11","slug":"microsoft-introduces-mu-the-slm-ai-in-windows","status":"publish","type":"post","link":"https:\/\/msftnewsnow.com\/microsoft-introduces-mu-the-slm-ai-in-windows\/","title":{"rendered":"Microsoft Mu Is The Super Tiny Language Model That Will Power AI in Windows Settings on Copilot+ PCs"},"content":{"rendered":"
\n
\n

Microsoft<\/a> has announced the introduction of Mu, a new on-device small language model (SLM<\/a>) designed to revolutionize the way users interact with Windows Settings on Copilot+ PCs. Developed by the Windows Applied Sciences team, Mu is engineered for high efficiency, low latency, and seamless integration with Neural Processing Units (NPUs), enabling a new class of AI-powered experiences directly on Windows devices.<\/span><\/p>\n

What Is Mu?<\/strong><\/h2>\n

\"Microsoft<\/p>\n

Mu is a compact, task-specific language model that runs entirely on-device, specifically leveraging the NPU hardware found in Copilot+ PCs. Unlike cloud-based models that require constant internet connectivity and can introduce latency, Mu operates locally, ensuring fast, private, and reliable responses to user queries within Windows Settings.<\/span><\/p>\n

The model is currently available to Windows Insiders in the Dev Channel<\/a> and is responsible for mapping natural language queries\u2014such as \u201cTurn on dark mode\u201d or \u201cIncrease screen brightness\u201d\u2014directly to the appropriate system settings actions.<\/span><\/p>\n

Technical Architecture and Efficiency<\/strong><\/h2>\n

Mu distinguishes itself through several technical innovations:<\/p>\n

    \n
  • \n

    Encoder-Decoder Transformer Architecture: <\/strong>Unlike traditional decoder-only models, Mu employs an encoder-decoder design. The encoder processes the input and generates a fixed-length latent representation, while the decoder uses this representation to generate the output. This separation allows Mu to reuse the encoded input, significantly reducing computational and memory overhead compared to decoder-only models.<\/span><\/p>\n

    \"Microsoft<\/h2>\n<\/li>\n
  • \n

    Optimized for NPUs: <\/strong>Mu\u2019s architecture and parameter shapes are meticulously tuned to align with the parallelism and memory constraints of NPUs. For example, the model uses a 2\/3\u20131\/3 split between encoder and decoder layers, maximizing performance per parameter. Weight sharing between input and output embeddings further reduces memory usage, a critical consideration for edge devices.<\/span><\/p>\n<\/li>\n

  • \n

    Performance Metrics: <\/strong>On a Qualcomm Hexagon NPU, Mu achieves 47% lower first-token latency and 4.7 times higher decoding speed compared to a similarly sized decoder-only model. This results in real-time responsiveness, with the model generating over 100 tokens per second\u2014essential for a smooth user experience in Windows Settings.<\/span><\/p>\n<\/li>\n<\/ul>\n

    Advanced Model Features<\/strong><\/h2>\n

    Mu incorporates several state-of-the-art transformer upgrades to maximize its efficiency and accuracy:<\/p>\n

      \n
    • \n

      Dual LayerNorm:<\/strong>
      \nNormalizing both before and after each sub-layer stabilizes training and keeps activations well-scaled with minimal overhead.<\/p>\n<\/li>\n

    • \n

      Rotary Positional Embeddings (RoPE):<\/strong>
      \nThese embeddings improve long-context reasoning, allowing Mu to handle sequences longer than those seen during training.<\/p>\n<\/li>\n

    • \n

      Grouped-Query Attention (GQA):<\/strong>
      \nGQA reduces the number of attention parameters and memory requirements while maintaining head diversity, decreasing latency and power consumption on NPUs.<\/span><\/p>\n<\/li>\n<\/ul>\n

      Training and Optimization<\/strong><\/h2>\n

      \"Microsoft<\/p>\n

      Mu was trained using A100 GPUs on Azure Machine Learning, following a multi-phase approach:<\/p>\n

        \n
      • \n

        Pre-training:<\/strong>
        \nThe model learned language syntax, grammar, semantics, and world knowledge from hundreds of billions of high-quality educational tokens.<\/p>\n<\/li>\n

      • \n

        Distillation:<\/strong>
        \nKnowledge was distilled from Microsoft\u2019s Phi models, enabling Mu to achieve remarkable parameter efficiency despite its small size.<\/p>\n<\/li>\n

      • \n

        Task-Specific Fine-Tuning:<\/strong>
        \nMu was further refined using datasets tailored to specific tasks, including SQUAD, CodeXGlue, and the Windows Settings agent. Fine-tuning with low-rank adaptation (LoRA) methods allowed Mu to deliver strong performance even with a micro-sized parameter count.<\/span><\/p>\n\n\n\n\n\n\n
        Task \\ Model<\/td>\nFine-tuned Mu<\/td>\nFine-tuned Phi<\/td>\n<\/tr>\n
        SQUAD<\/td>\n0.692<\/td>\n0.846<\/td>\n<\/tr>\n
        CodeXGlue<\/td>\n0.934<\/td>\n0.930<\/td>\n<\/tr>\n
        Settings Agent<\/td>\n0.738<\/td>\n0.815<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n<\/ul>\n

        Quantization and Hardware Collaboration<\/strong><\/h2>\n

        To ensure Mu runs efficiently on Copilot+ PCs, Microsoft applied advanced quantization techniques:<\/p>\n

          \n
        • \n

          Post-Training Quantization (PTQ):<\/strong>
          \nModel weights and activations were converted from floating point to 8-bit and 16-bit integer representations, drastically reducing memory and compute requirements without sacrificing accuracy.<\/p>\n<\/li>\n

        • \n

          Silicon Partner Collaboration:<\/strong>
          \nMicrosoft worked closely with AMD, Intel, and Qualcomm to optimize Mu for their respective NPUs, tuning mathematical operators and validating performance across different hardware platforms.<\/span><\/p>\n<\/li>\n<\/ul>\n

          The result is a model capable of producing outputs at more than 200 tokens per second on devices like the Surface Laptop 7, with ultra-fast response times even for large input contexts.<\/p>\n

          Mu in Action: The Windows Settings Agent<\/strong><\/h2>\n

          The primary application for Mu is the new AI-powered agent in Windows Settings. This agent is designed to simplify the process of changing system settings by understanding and executing natural language commands. For example, a user might type \u201cConnect to Wi-Fi\u201d or \u201cChange keyboard layout,\u201d and the agent will automatically navigate to the relevant setting and perform the requested action.<\/span><\/p>\n