Gemini can now see your screen and judge your tabs

(Image credit: Google)

Google is rolling out new vision features for Gemini Live
The AI assistant can now "see" your phone screen or camera feed
These upgrades are powered by Project Astra, Google's AI R&D umbrella

Google's Gemini Live is finally getting the gift of sight. The tech giant has quietly begun rolling out features that transform your humble smartphone into an all-seeing eye for its AI assistant.

The new abilities were uncovered by a Reddit user who later shared a video of the features in action. The upgrade lets Gemini peer through your screen or camera lens and process what it sees. The rollout marks the debut of Google's much-discussed and much-anticipated Project Astra.

Based on the video, Gemini's 'eyes' can analyze your screen in real-time through a "Share screen with Live" button. Gemini has long been able to digest static screenshots, but the update maintains a continuous gaze on your screen, looking at whatever you are on your phone for better or for worse.

Astra eyes

The new feature is arriving first for Gemini Advanced subscribers paying $20 a month for the Google One plan with extra AI. The rollout is notably democratic in where the feature appears, though, judging from the Xiaomi phone shown by the Reddit user. Google had previously hinted that Pixel and Galaxy S25 owners would have faster or better access to Project Astra.

Other AI assistants with similar seeing tools exist, but they are mostly tied to third-party apps like Microsoft Copilot, ChatGPT, Grok, and even Hugging Face's new HuggingSnap app. Having a real-time screen and camera-connect AI built into Android would certainly help entice users interested in an AI assistant to at least try Gemini.

And Google's timing in releasing the feature is notable as it tries to carve out a lead among AI assistants. Though Amazon has been hyping its new “Alexa Plus” update, it has yet to arrive.

Meanwhile, Apple's upgraded Siri has been delayed multiple times. That leaves Google with a temporary but very real lead in the AI assistant race. Gemini, for all its early hiccups and rebranding drama (RIP Bard), is now doing things that neither Alexa nor Siri can match for the moment.

Google has promised that Project Astra will be the "next-generation assistant" everyone wants to use all day. So keep your (and Gemini's) eyes peeled for new features to arrive in the weeks ahead.

See more Computing News

TOPICS

Eric Hal Schwartz is a freelance writer for TechRadar with more than 15 years of experience covering the intersection of the world and technology. For the last five years, he served as head writer for Voicebot.ai and was on the leading edge of reporting on generative AI and large language models. He's since become an expert on the products of generative AI models, such as OpenAI’s ChatGPT, Anthropic’s Claude, Google Gemini, and every other synthetic media tool. His experience runs the gamut of media, including print, digital, broadcast, and live events. Now, he's continuing to tell the stories people want and need to hear about the rapidly evolving AI space and its impact on their lives. Eric is based in New York City.