Microsoft Unveils Maia 300 AI Chip, Targets 40% Higher Tokens per Watt Than Leading GPUs

Microsoft has announced Maia 300, its third-generation in-house artificial intelligence (AI) chip, as the company looks to expand its AI computing infrastructure and reduce its reliance on external chip suppliers.

The chip is designed primarily for AI inference, the process through which trained AI models generate responses, analyse information and perform tasks. Microsoft plans to make its Maia chip family a primary inference system for its products and services, including its AI-powered platforms.

Microsoft CEO Satya Nadella said the company would begin scaling its Maia family to multi-gigawatt deployments from 2027, with the aim of supporting growing demand for AI computing.

Nadella also claimed that Maia 300 delivers 40% more tokens per watt than leading GPU-based systems. Tokens are the units of text that AI models process and generate, while tokens per watt measures how efficiently computing hardware produces them relative to power consumption.

Maia 300 Focuses on AI Inference Efficiency

Microsoft has designed Maia 300 around demanding AI workloads that require long context windows, substantial memory bandwidth and efficient processing of generated tokens.

One of its key features is support for the MXFP4 low-precision numerical format. This approach reduces memory bandwidth requirements for computation and key-value (KV) cache storage, potentially enabling larger context windows and higher throughput within the same power budget.

The chip also uses a software-defined dataflow architecture alongside AI-assisted kernel optimisation. These techniques are intended to reduce unnecessary data movement and improve the use of computing, memory and networking resources.

Microsoft aims to improve performance and power efficiency by optimising the entire system, from the processor and supporting infrastructure to the software running AI models.

Chiplet Design Offers Greater Flexibility

Maia 300 features a chiplet-based design that separates computing, memory control and input-output functions. This approach allows different components to be organised to meet the demands of specific workloads.

The architecture is part of Microsoft’s broader effort to develop an AI computing platform capable of supporting different models and applications across its cloud infrastructure.

The company said Maia chips are already being deployed across the United States, Europe and Australia. According to Nadella, existing deployments have delivered up to 1.7 times more tokens per watt across Microsoft Teams and Copilot experiences.

Microsoft Continues to Use Nvidia and AMD Chips

Developing its own AI processors gives Microsoft greater control over its computing infrastructure as demand for AI services grows. It could also help the company optimise costs, power consumption and performance across its data centres.

However, Maia 300 does not signal a complete shift away from third-party hardware. Nvidia and AMD processors will continue to be part of Microsoft’s computing resources as the company expands its in-house silicon programme.

The development could increase competition in the AI accelerator market, where Nvidia has a major presence and AMD is also expanding its offerings.

As Microsoft scales its Maia infrastructure, the company’s ability to deliver efficient AI inference at scale will be an important factor in determining how widely its in-house chips are adopted across its products and cloud services.