What is an NPU?

AI has been the biggest buzzword in technology since 2023, but it was earlier this year that ‘on-device AI’ became mainstream too. Unlike conventional cloud-powered AI, on-device artificial intelligence involves local processing and decision-making right on your smartphone or laptop. This is made possible by something called an NPU, a term you probably have come across.  

ALSO READ: What is an AI PC?

However, have you ever taken a step back and wondered what NPUs do? Here’s everything you need to know about what NPUs are, how they work, and what they can do for you. Read on. 

What is an NPU?

An NPU, or a Neural Processing Unit, is a dedicated processor on a larger chipset, especially designed for running neural network operations, which is what makes the AI features on our devices possible. Think of a brain with several interconnected neurons, that enables one to perform several tasks all at once. Well, NPUs work in a similar manner, wherein several interconnected smaller processing units on them work in parallel, which allows them to process large amounts of data in one go.  

This data is precisely what is used for real time speech or text input processing and generating responses to the same – whether in the form of text, speech, images or videos.

How does an NPU differ from a CPU or a GPU?

Now, contrary to popular belief, NPUs aren’t a recent invention. They’ve been around for years now. Apple, for example, started integrating NPUs into iPhones starting with the A11 Bionic chipset for the iPhone 8, 8 Plus and the iPhone X.  

Now, AI and Machine Learning tasks can be performed using GPUs and CPUs as well. However, not all GPUs are good for doing much, beyond processing graphics. CPUs are also the so-called “brains” of any given laptop and desktop, and work on processing several tasks parallelly, often not allowing them to function optimally for AI tasks.  

ALSO READ: ChatGPT vs Copilot vs Gemini: What’s different?

AI tasks are much more computationally intensive, and are therefore better handled by an NPU. A few of these tasks include videos and image generation, speech recognition, background blurring in video calls, photo or video editing processes such as object detection, among others.