Google is introducing powerful new AI-driven features to its Gemini Live assistant, enabling it to “see” what’s on a user’s screen or through their smartphone camera and respond in real time. This update, confirmed by Google spokesperson Alex Joseph in an email to The Verge, marks a major step in AI assistant capabilities. The underlying technology stems from “Project Astra,” which Google first showcased nearly a year ago.
Users are already starting to experience these features in action. A Reddit user recently reported that the screen-reading function appeared on their Xiaomi device, a discovery initially highlighted by 9to5Google. The same user later shared a video demonstrating Gemini’s ability to analyze on-screen content and provide relevant responses. This feature is part of Google’s broader plan to roll out advanced AI capabilities to Gemini Advanced subscribers through the Google One AI Premium plan, as originally announced in early March.
Alongside screen recognition, Google is also deploying live video interpretation. This allows Gemini to process a real-time feed from a smartphone camera and offer contextual insights. In a demonstration video shared by Google, a user utilizes the feature to seek advice on choosing a paint color for their freshly glazed pottery, showcasing the AI’s potential in everyday decision-making.
Google’s latest AI rollout highlights its continued dominance in the AI assistant space. While Amazon is gearing up for an early-access launch of its upgraded Alexa Plus and Apple has postponed its revamped Siri, Google is already delivering next-generation AI tools to users. Samsung’s Bixby remains an alternative, but Gemini continues to be the default assistant on Samsung devices, reinforcing Google’s leadership in AI-powered virtual assistants.
