Top 10 AI Chip Makers of 2024: In-depth Guide

As the figure above illustrates, the number of parameters (consequently the width and depth) of the neural networks and therefore the model size is increasing. To build better deep learning models and power generative AI applications, organizations require increased computing power and memory bandwidth. 

Powerful general-purpose chips (such as CPUs) cannot support highly parallelized deep learning models. Therefore, AI chips that enable parallel computing capabilities are increasingly in demand, and according to McKinsey, this trend will continue.

However, even Intel, which has numerous world-class engineers and a strong research background, needed three years of work to develop its own AI chip. Therefore, for most companies, buying chips from these vendors or renting capacity from cloud GPU providers is the only way to develop powerful deep learning models. This article will introduce top AI chip vendors to help companies choose the right one.

Which are the leading AI chip producers?

1. Nvidia

Nvidia has been producing graphics processing units (GPUs) for the gaming sector since 1990s. The PlayStation 3 and Xbox both use Nvidia graphics arrays. The company also makes AI chips such as Volta, Xavier, and Tesla. Thanks to the generative AI boom, NVIDIA had excellent results in Q2 2023, reached a trillion in valuation and solidified its status as the leader of GPU and AI hardware markets.

NVIDIA’s chipsets are designed to solve business problems in various industries. Xavier, for example, is the basis for an autonomous driving solution, while Volta is aimed at data centers. DGX™ A100 and H100 are the flagship AI chips of Nvidia which are designed for AI training and inference in data centers. A100 integrates 8 GPUs and up to 640GB GPU memory. Nvidia Grace is the new AI chip model that the company released for the HPC market in 2023.

For AI workloads on the cloud, Nvidia almost has a monopoly with most cloud players offering only Nvidia GPUs as cloud GPUs. Nvidia also launched its DGX Cloud offering providing cloud GPU infrastructure directly to enterprises.

2. Advanced Micro Devices (AMD)

AMD is a chip manufacturer that has CPU, GPU and AI accelerator products. For instance, AMD’s Alveo U50 data center accelerator card has 50 billion transistors. Accelerator can run 10 million embedding datasets and perform graph algorithms in milliseconds.

AMD launched MI300 for AI training workloads in June 2023 and will be competing with NVIDIA for marketshare in that market.¹ There are startups, research institutes, enterprises and tech giants that adopted AMD hardware in 2023 since Nvidia AI hardware has been difficult to procure due to rapidly increasing demand with the rise of generative AI as demonstrated by ChatGPT.²³⁴⁵

AMD is also working with machine learning companies like Hugging Face to enable data scientists to use their hardware more efficiently.⁶

The software ecosystem is critical as hardware performance relies heavily on software optimization. For example, AMD and NVIDIA had a public disagreement over benchmarking H100 and MI300. The focus of the disagreement was the package and floating point to use in the benchmark. According to the latest benchmarks, it appears that MI300 is better or on par with H100 for inferencing on a 70B LLM.⁷⁸

3. Intel

Intel is the largest player in the CPU market and has a long history of semiconductor development. In 2017, Intel became the first AI chip company in the world to break the $1 billion sales barrier. 

Intel’s Xeon CPUs are appropriate for a variety of jobs, including processing in data centers and have had an impact on the company’s commercial success.

Gaudi3 is the latest AI accelerator processor from Intel. Since it will be publicly released in 2024, there are currently limited benchmarks on its performance.⁹

Which public cloud providers produce AI chips?

4. Alphabet / Google Cloud Platform

Google Cloud TPU is the purpose-built machine learning accelerator chip that powers Google products like Translate, Photos, Search, Assistant, and Gmail. It can be used via the Google Cloud as well. Google announced TPUs in 2016.¹⁰

Edge TPU, another accelerator chip from Google Alphabet, is smaller than a one-cent coin and is designed for edge devices such as smartphones, tablets, and IoT devices.

5. AWS

AWS produces Tranium chips for model training and Inferentia chips for inference. Though AWS is the market leader in public cloud, it started building its own chips after Google.

6. IBM

IBM announced its latest deep learning chip, artificial intelligence unit (AIU), in 2022.¹¹. IBM is considering using these chips to power its watson.x generative AI platform.¹²

AIU builds on “IBM Telum Processor” which powers AI processing capabilities of IBM Z mainframe servers. At launch, Telum processors’ highlighted use cases included fraud detection.¹³

IBM also demonstrated that merging compute and memory can lead to efficiencies. These were demonstrated in the NorthPole processor prototype.¹⁴

7. Alibaba

Alibaba produces chips like Hanguang 800 for inference. However, some North American, European and Australian organizations (e.g. those in the defense industry) may not prefer to use Alibaba Cloud for geopolitical reasons.

Who are the leading AI chip startups?

We would also like to introduce some startups in the AI chip industry whose names we may hear more often in the near future. Even though these companies were founded only recently, they have already raised millions of dollars.

8. SambaNova Systems

SambaNova Systems was founded in 2017 with the goal of developing high-performance, high-precision hardware-software systems for high volume generative AI workloads. The company has developed the SN40L chip and raised more than $1.1 billion in funding.¹⁷¹⁸

It is important to note that SambaNova Systems also leases its platform to businesses. AI platform as service approach of SambaNova Systems makes their systems easier to adopt and encourages hardware reuse for circular economy. 

9. Cerebras Systems

Cerebras Systems was founded in 2015. In April 2021, the company announced its new AI chip model, Cerebras WSE-2, which has 850,000 cores and 2.6 trillion transistors. Undoubtedly, the WSE-2 is a big improvement over the WSE-1, which has 1.2 trillion transistors and 400,000 processing cores. 

Celebra’s system works with many pharmaceutical companies such as AstraZeneca and GlaxoSmithKline because the effective technology of WSE-1 accelerates genetic and genomic research and shortens the time for drug discovery.

10. Groq

Groq has been founded by former Google employees. The company represents LPUs, a new model for AI chip architecture, that aims to make it easier for companies to adopt their systems. The startup has already raised around $350 million and produced its first models such as GroqChip™ Processor, GroqCard™ Accelerator, etc.

The company is focused on LLM inference and released benchmarks for Llama-2 70B.¹⁹

In Q1 2024, the company shared that 70k developer signed up on its cloud platform and built 19k new applications.²⁰

On the March 1, 2022, Groq acquired Maxeler, which has high performance computing (HPC) solutions for financial services.

What are upcoming AI hardware producers?

Though these are compelling AI hardware solutions, there are currently limited benchmarks on their effectiveness since they are newcomers to the market.

Meta

Meta Training and Inference Accelerator (MTIA) is a family of processors for AI workloads such as training Meta’s LLaMa models.

The latest model is Next Gen MTIA which is based on TSMC 5nm technology and is claimed to have 3x improved performance vs. MTIA v1. MTIA will be hosted in racks containing up to 72 accelerators.²¹

MTIA is currently for Meta’s internal usage. However, in the future, if Meta launched a LLaMa based enterprise generative AI offering, these chips could power such an offering.

Microsoft Azure

Microsoft launched Maia AI Accelerator in November 2023²²

Rebellions

Korea based startup raised $124M in 2024 and is focused on LLM inference.²³

What are other AI chip producers?

Graphcore

Graphcore is a British company founded in 2016. The company announced its flagship AI chip as IPU-POD256. Graphcore has already been funded with around $700 million.

Company has strategic partnerships between data storage corporations like DDN, Pure Storage and Vast Data. Graphcore works with research institutes around the globe like Oxford-Man Institute of Quantitative Finance, University of Bristol and Berkeley University of California are other reputable research organizations that use Graphcore’s AI chips.

As of October 2023, the company’s long term viability is at risk as it is losing ~$200M per year and ~$160M in assets as of Jan 1st, 2023.²⁴²⁵

Mythic

Mythic was founded in 2012 and is focused on edge AI. Mythic follows an unconventional path, an analog compute architecture, that aims to deliver power-efficient edge AI computing.

It developed products such as M1076 AMP, MM1076 key card, etc., and has already raised about $165 million in funding.²⁶

Mythic laid of most of its staff and restructured its business with its funding round in March 2023.²⁷

What are companies reported to be working on AI hardware?

These companies are yet to launch their AI hardware.

_etched

The team claims to have built the world’s first transformer supercomputer however AIMultiple team has not yet come across any benchmarks or client references.

OpenAI

OpenAI is reported to be raising funds to build its own AI hardware. ²⁸

You can also check our sortable list of companies working on AI chips.

You might enjoy reading our articles on TinyML and accelerated computing.

If you have questions about how AI hardware can help your business, we can help:

References

• 1. “AMD MI300“. Semianalysis. June 12, 2023. Retrieved June 18, 2023.
• 2. “Lamini & AMD: Paving the Road to GPU-Rich Enterprise LLMs“. Lamini. August 16, 2023. Retrieved October 12, 2023.
• 3. “Announcing AI2 OLMo, an Open Language Model Made by Scientists, for Scientists“. Allen Institute for AI. May 11, 2023. Retrieved November 1, 2023.
• 4. “Training LLMs at Scale with AMD MI250 GPUs“. Databricks. October 30, 2023. Retrieved November 1, 2023.
• 5. “TRAINING 221B PARAMETER KOREAN LLM ON 1,200 AMD MI250 GPU CLUSTER“. MOREH. August 14, 2023. Retrieved November 1, 2023.
• 6. “AMD + 🤗: Large Language Models Out-of-the-Box Acceleration with AMD GPU“. 🤗. December 5, 2023. Retrieved December 29, 2023.
• 7. “Achieving Top Inference Performance with the NVIDIA H100 Tensor Core GPU and NVIDIA TensorRT-LLM“. NVIDIA. December 14, 2023. Retrieved December 29, 2023.
• 8. “Competitive performance claims and industry leading Inference performance on AMD Instinct MI300X“. AMD. December 15, 2023. Retrieved December 29, 2023.
• 9. “Intel Accelerates AI Everywhere with Launch of Powerful Next-Gen Products“. Intel. December 14, 2023. Retrieved January 1, 2024.
• 10. “Google supercharges machine learning tasks with TPU custom chip“. Alphabet. May 19, 2016. October 8, 2023
• 11. “Meet the IBM Artificial Intelligence Unit“. IBM. October 18, 2022. Retrieved October 2, 2023
• 12. “IBM to use in-house AI chip to reduce costs of Watsonx“. T_HQ. July 12, 2023. Retrieved October 2, 2023
• 13. “IBM’s newest chip is more than meets the AI“. IBM. August 23, 2021. Retrieved October 2, 2023
• 14. “‘Mind-blowing’ IBM chip speeds up AI“. Nature. October 19, 2023. Retrieved November 12, 2023.
• 15. “Most funded artificial intelligence (AI) chip startup companies worldwide 2021.” Statista. Revisited October 9, 2023.
• 16. Nellis, S (2021).”AI chip startup Cerebras Systems raises $250 million in funding.” Reuters. Revisited October 9, 2023.
• 17. “SambaNova Unveils New AI Chip, the SN40L, Powering its Full Stack AI Platform“. SambaNova. September 19, 2023. Retrieved October 2, 2023. SambaNova. Retrieved October 2, 2023.
• 18. “SambaNova Systems“. Crunchbase. Retrieved October 2, 2023.
• 19. “Groq Sets New Large Language Model Performance Record of 300 Tokens per Second per User on Meta AI Foundational LLM, Llama-2 70B”. PR Newswire. November 7, 2023. Retrieved November 28, 2023.
• 20. Demand for Real-time AI Inference from Groq® Accelerates Week Over Week. Groq. April 2, 2024
• 21. “Our next-generation Meta Training and Inference Accelerator“. Meta. April 10, 2024
• 22. “With a systems approach to chips, Microsoft aims to tailor everything ‘from silicon to service’ to meet AI demand“. Microsoft. November 15, 2023. Accessed December 25, 2023.
• 23. “Korean AI chipmaker Rebellions Closes $124M Series B Fundraise“. Rebellions. January 30, 2024. Accessed February 1, 2024.
• 24. “Graphcore limited“. Gov.uk. Retrieved October 8, 2023
• 25. “Graphcore limited annual report and financial statements“. Graphcore. Retrieved October 8, 2023
• 26. “Mythic“. dealroom.co. Retrieved October 25, 2023.
• 27. “AI chip startup Mythic rises from the ashes with $13M, new CEO“. Techcrunch. Retrieved October 25, 2023.
• 28. “OpenAI in talks to raise new funding at $100 bln valuation – Bloomberg News“. Reuters. December 23, 2023. Accessed December 25, 2023.