Microsoft’s Maia 300 aims to squeeze more AI out of every megawatt

Microsoft has unveiled Maia 300, its latest custom artificial intelligence (AI) chip, as the technology giant steps up efforts to build its own AI infrastructure and reduce its reliance on third-party processors from companies such as Nvidia and AMD.

Announced at the OCP Global Summit 2026 in San Jose on Monday, October 12, Maia 300 is designed to handle increasingly demanding AI inference workloads, including long-context reasoning, conversational applications and multi-step tasks performed by AI agents.

Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than leading graphics processing unit (GPU)-based systems. A rack containing 96 accelerators is also expected to deliver up to seven times the rack-scale token throughput of its predecessor, Maia 200.

The company plans to begin production deployments of Maia 300 in 2027, when it also intends to scale its Azure Maia family to multi-gigawatt deployments. Microsoft expects Maia 300 to become the primary inference system for its products and services.

Maia 200 already powers Microsoft’s AI workloads

While Maia 300 is still some time away from production deployment, its predecessor is already being used across Microsoft’s AI infrastructure.

The company said Maia 200 is serving production workloads, including Microsoft Copilot, while supporting its Microsoft AI (MAI) team in developing next-generation models. GitHub Copilot and Microsoft Foundry are expected to follow by the end of 2026.

Maia 200 supports OpenAI models, Microsoft’s own MAI models and leading open-source model families. The chip is being deployed across the United States, Europe and Australia, according to the company.

Microsoft also said Maia 200 delivers up to 1.7 times more tokens per watt for built-in AI capabilities in Microsoft Teams, Copilot and Cowork experiences than leading GPU infrastructure.

Tokens are the basic units of data processed by AI models. They can represent whole words, parts of words or other data elements. The number of tokens a system processes relative to the energy it consumes is one measure of its efficiency, particularly as companies look to serve more AI queries without proportionately increasing their power requirements.

Microsoft said its experience running models and services on Maia 200 has directly informed the design of Maia 300.

Why Microsoft is building its own AI chips

Microsoft’s latest announcement comes as major technology companies seek greater control over the hardware underpinning their AI services. Amazon, Google and Meta have developed or are developing custom chips, while AI companies such as OpenAI and Anthropic are also exploring alternatives to relying entirely on external processors.

Developing in-house silicon allows companies to tailor chips to their workloads, potentially improving performance, energy efficiency and operating costs. It can also help reduce exposure to supply constraints and dependence on a handful of semiconductor suppliers.

For Nvidia, one of the leading suppliers of AI accelerators, the expansion of custom silicon programmes among its largest customers presents a potential competitive challenge. Hyperscalers are major buyers of its processors, and their growing ability to design chips for specific workloads could influence how they allocate future spending.

However, the shift towards in-house chips does not necessarily mean demand for Nvidia’s processors will disappear. Large AI infrastructure operators continue to require different types of hardware for different workloads, and custom accelerators will have to compete with established GPU platforms on performance, software compatibility and availability.

Maia 300 targets memory and power efficiency

Microsoft has designed Maia 300 specifically for inference workloads. Inference involves running trained AI models to generate responses, process information or carry out tasks. These workloads are becoming increasingly important as AI moves beyond basic chatbot interactions towards systems capable of reasoning, planning and completing multi-step tasks.

The chip uses eight stacks of high-bandwidth memory (HBM4), delivering peak memory bandwidth of 18 terabytes per second within a 950-watt thermal design power envelope. Microsoft said the architecture is designed to reduce unnecessary data movement and improve the utilisation of computing, memory and networking resources.

Maia 300 also uses a multi-chiplet design that separates computing, memory-controller and input/output functions. Its compute chiplets are built using Taiwan Semiconductor Manufacturing Company’s (TSMC) N2P process technology.

At the rack level, a 96-accelerator configuration combines 27.6 terabytes of HBM capacity with aggregate memory bandwidth of 1.7 petabytes per second, within a 167-kilowatt power envelope.

These specifications are intended to help Microsoft handle more simultaneous AI interactions and larger workloads within a given datacentre footprint. The company is also optimising the system for larger context windows, which allow AI models to process more information during an interaction, and for the networking needed to coordinate work across multiple accelerators.

What Maia 300 means for Microsoft’s AI ambitions

Maia 300 is expected to support a range of Microsoft’s AI services, including coding tools, cloud-based model deployment through Microsoft Foundry, model development by the MAI team and clinical workflow applications through Dragon Copilot.

The broader objective is to bring Microsoft’s chips, software and AI services closer together, allowing the company to optimise its infrastructure for the workloads it expects to run at scale.

The commercial significance of that strategy will depend on how effectively Maia chips perform in real-world deployments and whether their efficiency gains translate into lower costs as AI demand expands.

For now, Microsoft is positioning its custom silicon as an increasingly important part of its AI infrastructure, rather than signalling an end to its reliance on the wider AI chip market.