How Google is using AI to upgrade the Android experience

If there was one hot topic at the Google I/O 2024 keynote, it was AI. With the company making waves on the AI front for years now, Gemini being its latest consumer-facing end-product, it was only a matter of time before Google made AI more personal. Now, that approach is taking form via Google’s future plans for Android phones.

During the keynote, Google made it clear that it wants Android to have AI at its core. The new features announced, give Google the edge against Apple’s iOS, well, at least until iOS drops its own AI innovations next month at WWDC 24.

ALSO READ: Google I/O 2024: Top 5 AI announcements

Circling back to Mountain View, Google’s AI-fueled plans for Android kick off with three key features this year. Here’s a quick look at them.

Circle to Search

Yes, Google’s Circle to Search feature isn’t exactly new, and we already had a peek at it during Samsung’s launch of the Galaxy S24 series. However, Google is set on bringing Circle to Search to more devices, while also improving the capabilities of the feature moving forward.

For the uninitiated, Circle to Search is a feature that captures anything on the screen, including snips from third-party apps, and allows users to highlight particular elements. Google will then intelligently comprehend what you’re ‘pointing’ at and deliver helpful information in seconds.

Circle to Search could be used to pull up a celebrity and find out what shoes they are wearing, or it can be used to understand what monument stands behind your neighbour in their newest pictures from a trip. You can understand how the feature works in the video below.

Gemini AI Assistant

Google also announced that its primary AI tool – Gemini, will soon be becoming the default AI assistant on Android phones, implying that it will be replacing trusty old Google Assistant. Worry not though, Gemini is more than likely to pack all of Google Assistant’s capabilities, but with the ability to do a lot more.

This includes using generative AI for text or images on the fly, or any of the other cool new AI tricks Google showcased during its keynote. This will also include Gemini being able to use the camera feed in real-time to display helpful information on anything that’s in front of you.

ALSO READ: Google Veo is a text-to-video AI model that is here to take on OpenAI’s Sora

Lastly, making Gemini the default assistant on Android is also a smarter move in the long run, as it will become a central tool for Google to pour all its future AI innovations into. This will also make the entire AI experience easier for users, who will find all the cool new features in one single tool, that will also happen to be integrated on a system-level across their smartphones.

On-Device AI

The third aspect Google is homing in on when it comes to Android AI innovation, on-device AI. Google announced that Android will soon be the first operating system that will feature a built-in, on-device foundation model. Think of this as a middle-man that brings the capabilities of Gemini together with an NPU-powered chip, to ensure a fast and safe AI experience.

Fast, because with on-device AI, you rely lesser on cloud-based responses, allowing Gemini to natively run on your phone and deliver results much more quickly. Safe, because lesser information going to the cloud, also means lesser information leaving your device to begin with. The latter, is imperative with multi-modal AI becoming a big part of Google’s plans, and will ensure inputs from your phone’s camera or mic stay secure.

Google also announced that it will soon bring in Gemini Nano with multi-modality to Android, starting with Pixel phones. Working very similar to the Project Astra demo, the feature will allow the Gemini app to work on text, video, and audio inputs simultaneously, all without sensitive data leaving the device.

ALSO READ: Google’s Project Astra explained

In a nutshell, Android is quickly looking like the main platform for Google to bring all its AI innovations to. Very soon, Android could be a much more capable operating system, with AI features baked in, changing how we use our phone for day-to-day tasks, from the mundane to the creative!