• by Laurie Sullivan  @lauriesullivan, August 13, 2024

Google's dominance in advertising and search took years of software support -- to build performance and innovative services embedded in the hardware or supported by the cloud. 

Mobile phones eventually became consumers' computers, which can be carried in a pocket or handbag.

On Monday, through its Made By Google brand, Google launched a new line of phones with custom semiconductor chips and AI-powered Gemini upgrades supported by features on the phone or in the cloud. All the phones — Pixel 9, Pixel 9 Pro and Pixel 9 Pro XL  — are powered by Google's Tensor G4 custom silicon chip.

Tensor G4 was designed with Google DeepMind to open pages more quickly and process photographs, images and text, and is optimized to run Google’s most advanced AI models. It is now the first processor to run Gemini Nano with multimodality, which helps the phone understand text, images and audio.

Google has also advanced its subscription services. Pixel 9 phones are the first to be developed and designed for the Gemini era.

During the event, Google announced Gemini Live, an AI assistant that has the option to personalize the voice from preset choices. It allows users to have free-flowing conversations with the ability to change the topic in midstream, but it is only available to Gemini Advanced subscribers.

Google has made decades of investments in AI. The latest version of Gemini will be available in all Pixel phones, and supported in 45 languages. It can reason, draft messages, brainstorm, and provide personalized support by accessing apps across a phone running android when given permission.

Gemini Live is only available as a paid subscription service.

On devices, Gemini Nano enables the same type of support, and the data never leaves the phone.

Dave Citron, senior director of product, Gemini at Google, attempted to demonstrate during Made by Google how Gemini can check his calendar to see if he is available to meet with someone based on a photograph of that friend’s schedule. As with many live demos during a large event, Gemini failed on the first two demonstrations, but was successful in the third demonstration.

There were many instances when the demonstrations went without a hitch. Some of them included Pixel Screenshots that can extract information within an image, and a weather app that provides information about UV settings and more. Both use Gemini.

Call Assist now offers Call Notes, which can create a private summary of conversations on the phone.

Google says the tool is completely private, because the audio never gets sent to the cloud, but it transcribes and summarizes calls, keeps it local on the phone.

Google rebuilt Panorama to provide detailed shots, even in low light. and Magic Editor in Google Photos has new editing capabilities. Auto frame allows the photographer reframe a photo for better composition, or to reimagine photos by typing what should be in the photo, but never made it into the frame -- for example, adding wildflowers to a field of grass. Although this potential may seem a little disturbing, virtual reality can make it happen.