• by Laurie Sullivan  @lauriesullivan, December 13, 2024

Google has launched an early prototype of an AI agent called Project Mariner.

The AI agent, which explores the interaction in the browser between humans and technology, is built on Gemini 2.0.

It can understand and reason across information, including the ability to identify pixels and web elements like text, code, images and forms. It then uses the information with an experimental Chrome extension to complete tasks.

The company said the tests include conducting active research on new types of risks and mitigations.

For example, if a computer has five tabs open in a browser, Project Mariner can only type, scroll or click in the active tab. It asks users for final confirmation before taking certain sensitive actions, such as making a purchase.

Project Mariner can fill the online shopping basket, but the human consumer needs to confirm the purchase. 

"It’s still early, but Project Mariner shows that it’s becoming technically possible to navigate within a browser, even though it’s not always accurate and slow to complete tasks today, which will improve rapidly over time," Google writes in a blog post. 

In a test, Google added a list of companies into the prompt and asked the agent to take the list and look up their websites and look up content information to reach someone. It begin by searching Google and clicks into the website.

At any point in the process, the user can stop the agent or pause the action. It also lets the user see the reasoning in the interface to better understand what it is doing. It records information as it goes.

Pichai also introduced Google's next era of models -- Gemini 2.0 -- built for the agentic era.

The model is a type of AI system that acts autonomously on behalf of its users, making decisions and taking actions to achieve goals. The model includes advancements in multimodality such as native image and audio output, and native tool use.

Pichai explained how this technology will enable Google to build new AI agents that bring the company closer to its vision of a universal assistant.

One of those being tested is Project Mariner. Testers have begun testing the AI agent using an experimental Chrome extension, and conversations are happening with companies across the web to explore possibilities of this technology.

Google said when evaluated against the WebVoyager benchmark, which tests agent performance on end-to-end real world web tasks, Project Mariner achieved a state-of-the-art result of 83.5% working as a single agent setup.