• by Laurie Sullivan , Staff Writer @lauriesullivan, February 21, 2024

A few months after Google DeepMind unveiled Gemini, which it calls its most capable AI model, the company released a compact offspring it named Gemma, described as a series of "lightweight" open AI models designed for text generation and other language tasks.

The AI models are geared toward developers who don't need or can't afford to use the biggest AI models such as Gemini.

Gemma is a family of lightweight, state-of-the-art open AI models built from the same research and technology used to create Gemini. The models are developed by Google DeepMind and other teams across Google.

Google says the Gemma models are capable of running directly on a developer’s laptop or desktop computer. It can surpass significantly larger models on key benchmarks while adhering to standards for safe and responsible outputs.

The name reflects the Latin meaning of the word gemma, “precious stone.” Accompanying the models being released in two sizes —  Gemma 2B and Gemma 7B — a responsible generative AI (GAI) toolkit to provide guidance for creating safer AI applications, pre-trained instructions, ways to optimize across AI hardware platforms like Nvidia GPUs and Google Cloud TPUs, and more.

The technical report explains more. The report details the model's architecture and training infrastructure. it offers "fine-tuning recipes" for Gemma, followed by evaluations of all checkpoints across a wide-variety of quantitative and qualitative benchmarks, as well as both standard academic benchmarks and human-preference evaluations.

The authors of the report then detail their approach to safe and responsible deployment, and outlines the broader implications of Gemma, its limitations and advantages, and conclusions.

Yes Gemma has benefits and limitations, the researchers acknowledge. To ensure transparency for users, Google published a detailed model card to provide researchers with a comprehensive understanding of the model.

The benefits of Gemma, researchers say, include code, data, architecture, instruction tuning, reinforcement learning from human feedback, and evaluations. But they agree that further research is needed to create safe models that reliably perform as intended. Example further research areas include factuality, alignment, complex reasoning, and robustness to adversarial input. As discussed by Gemini, we note the need for more challenging and robust benchmarks.

No doubt, Google is seeding the enterprise market to build applications that run on Gemma similar to the way apps were built.

Microsoft has also invested in the market for smaller models, via Phi range. In December released news on the small language model that the company has been working to train on much smaller data sets—“textbook-quality”—data. In a research blog, at the time, Microsoft said Phi-2 continues to perform on par or better than certain larger open-source Llama 2 models, including one with 13 billion parameters.