Microsoft Ventures Into Custom Silicon With New AI Accelerator and CPU
Microsoft is making its debut into in-house silicon with two new custom chips, claimed to improve AI performance in its data centers.
Microsoft has unveiled two new custom-designed chips to accelerate its unique AI and compute workloads in its servers and data centers. The two chips, revealed at Microsoft Ignite, represent the culmination of Microsoft’s efforts to bring the entirety of its server architecture to in-house development.
Microsoft tests its custom-designed chips in-house, bringing more control over compute performance and efficiency for AI and data center applications.
Many organizations, including Amazon and Google, have turned to custom chip development to bolster their computing performance. Microsoft’s announcement continues this trend, and will eventually offer more data points to evaluate the tradeoffs between custom and commercial silicon.
Introducing the Maia AI Accelerator and Azure Cobalt CPU
Microsoft’s two new pieces of custom silicon are the Maia AI Accelerator and the Azure Cobalt CPU. Each will eventually play a critical role in Microsoft’s custom-built Copilot and Azure OpenAI servers as developers demand more performance for cloud-based computing.
The Maia 100 chip offers higher performance than commercial chips, allowing Microsoft to optimize its own unique workloads and offer a fully custom solution.
Microsoft doesn’t currently have a comprehensive spec sheet for the Maia and Cobalt chips. We do know, however, that each chip is built on a TSMC 5 nm process, with the Maia AI accelerator including over 100 billion transistors and a 4.8 Tbps per accelerator Ethernet-based protocol.
The Cobalt CPU, on the other hand, is an Arm-based 64-bit, 128-core processor built to complement the Maia GPU in parallel processing tasks. The Cobalt chip reportedly offers a 40% performance improvement over current-generation Azure servers, giving designers access to higher compute performance.
Building Bespoke Servers
With Microsoft’s venture into custom silicon, the company joins others, such as Amazon and Google, who have abandoned commercial chips for purpose-built chips to optimize performance. Especially in the case of cloud servers running advanced AI models, the computation and efficiency improvements of custom silicon can greatly impact the overall performance of a data center.
In Microsoft’s case, the jump to custom silicon is the final component needed for fully custom servers. Microsoft has even adopted a unique approach to cooling the Maia chips. Instead of a large liquid chiller, Microsoft employs custom server racks with space for the compute card and a “sidekick” that contains the liquid cooling mechanism.
An example of the Maia server highlights Microsoft’s fully custom approach. From server rack to liquid cooling “sidekick”, the Maia server includes all Microsoft-designed parts.
Throughout the process, Microsoft has worked with OpenAI to provide feedback on the Maia and Cobalt chips’ performance with large language models. Although the Maia chips may provide higher performance, Microsoft will still use commercial silicon to create a customer-focused solution, adding more versatility to its offerings.
The Trend Toward Custom Silicon
As more companies move toward custom silicon in their HPC and data center offerings, AI developers are reaping the benefits of improved computing power. Companies can also customize their solutions to find the best operating point for efficiency, computing power, and versatility in their cloud computing centers.
Despite the fact that Maia and Cobalt were only recently announced, Microsoft has reported that it is already designing second generations of the chips, demonstrating how custom silicon allows developers to precisely control development times without relying on third parties.
All images (modified) used courtesy of Microsoft.
