Skip to content
VECTOR WIREAI INTELLIGENCE
UTC
Home

Google extends Gemini on Mac with computer-use agent for local apps and web

Gemini Mac agent rolls out to US Google AI Plus, Pro, and Ultra subscribers in English.

7Gemini on Mac can now use your computertheverge.comHigh11·19 18:21Z

Google on November 19 added computer-use capabilities to the Gemini desktop application on macOS, enabling the AI to click, type, navigate menus, and fill out forms across local desktop apps, files, and the web on a user's behalf1,4,5. The feature is rolling out in English to users in the United States on Google AI Plus, Pro, and Ultra tiers6,7. Users invoke the agent by adding "@Computer use" to their prompts3.

How the agent works

Gemini operates inside an isolated background window with a live preview displayed in the user's chat, leaving the Mac's main display and cursor free for other work. Users configure which apps Gemini can access, can pause or stop the agent at any time with one click, and must approve sensitive actions before they execute. Google pauses automatically for logins and payments, requiring explicit confirmation.

The assistant supports tasks comprising dozens of actions, and users can feed the output of one completed workflow into a follow-up project. Google's Michael Friedman wrote that the feature "saves you time on the repetitive work that normally takes dozens of clicks" and that users need only describe what they need before Gemini "handles the clicking, typing, menus, and forms for you". Gemini can interact with both local apps and web services, loading data from webpages into desktop programs or moving data in the opposite direction and between cloud services.

Use cases and subscription requirement

Google highlighted several workflows: batch color-grading photos in design software, extracting weights and dimensions from local PDFs to fill multi-box shipping quotes, turning live data into slide decks from screenshots, and batch-listing items for sale by analyzing photos in a local folder, researching pricing, and staging listings on resale platforms. A designer could upload a vehicle sketch and ask Gemini to turn it into a three-dimensional animation, with the agent finding the needed local app, creating a project plan, and carrying out the work step by step.

The feature requires a Google AI subscription, which equips consumer Google accounts with a broader lineup of AI tools and additional storage.

Apple's new permission controls

Google's announcement came three days after Apple introduced tighter permission controls for macOS automation, according to The New Stack2. The macOS 27.3 beta 2 changes split Device Control and Data Access permissions into two categories: System App Protection Override and Accessibility. Apps requesting broad system access face more explicit permission prompts, and users can approve individual requests instead of granting ongoing access. Google has not explained how Gemini will operate under the new restrictions.

Competitive context

The launch follows similar agentic desktop features already offered by OpenAI, Anthropic, and Meta. Google itself only recently began the broader rollout of Gemini's Tasks, the in-app feature that lets the AI sign into websites, make online purchases, or book appointments. Google also recently began admitting early testers to its revamped CC AI agent, announced in September, which is designed to help families stay organized across email, calendars, chats, and tasks. Google separately built an isolated synthetic web replica to train and evaluate the AI browsing agents powering its auto-browse feature[3].

ANALYSIS Computer use on macOS closes a visible gap between Gemini's desktop agent and the agentic desktop tools already shipping from OpenAI, Anthropic, and Meta, though the initial rollout is limited to US English-language users on paid tiers. The unresolved question of how Gemini will handle Apple's freshly tightened automation permissions adds a practical uncertainty for users considering adoption.

Was this useful?

The day’s vector, in your inbox.

Vector Wire Daily: one dispatch each publishing day.