Google on KnightLi Blog

How to Use Gemini 3.5 Flash and Gemini Omni for Free: Entry Points for Users and Developers

Wed, 20 May 2026 23:13:35 +0800

After Google released Gemini 3.5 Flash and Gemini Omni, the practical question is not the benchmark score, but how ordinary users and developers can actually use them, which entry points are free, and which ones are only low-friction trials.

The short version:

For chat, writing, image understanding, and everyday Q&A: use Gemini app first.
To test Gemini 3.5 Flash parameters, prompts, and multimodal input: use Google AI Studio.
To call Gemini 3.5 Flash from code: create an API key in AI Studio.
To try it from the terminal for free: look at Gemini CLI.
To try Gemini Omni video editing: start with Gemini app and Google Flow.
For real production use: do not rely on free quotas; move to a paid API or Vertex AI.

Note: free quotas, regional availability, subscription tiers, and model menus change over time. This article was written on May 20, 2026. Before official use, always check Google’s current pages.

Free Gemini 3.5 Flash Method 1: Gemini App

The simplest entry point is Gemini app:

https://gemini.google.com/

The basic flow is straightforward:

Open Gemini.
Sign in with a Google account.
Look for 3.5 Flash in the model selector.
Start chatting.

This entry point is best for ordinary users. You can use it for writing, summarization, image understanding, file analysis, everyday Q&A, and simple planning. According to public reports, Gemini 3.5 Flash has been made available to users globally and can be selected from Gemini’s model dropdown.

The limits are also clear: free users usually face daily message, regional, and feature limits. If you exceed the limit, you need to wait for the quota to refresh or upgrade your subscription.

Free Gemini 3.5 Flash Method 2: Google AI Studio

If you want more than chat, and need to tune prompts, inspect parameters, or test structured output, Google AI Studio is a better fit:

https://aistudio.google.com/

Basic flow:

Sign in to Google AI Studio.
Create a new prompt.
Select gemini-3.5-flash in the model dropdown.
Enter the prompt and run it.

AI Studio gives you more control. You can adjust temperature, system instructions, structured output, and multi-image input, and you can export a working prompt into code or an API call.

For developers, AI Studio is a free testing bench. Tune the prompt and input format here first, then move into API integration to avoid wasting quota.

Free Gemini 3.5 Flash Method 3: Free API Key

Developers care most about the API. AI Studio can create a Gemini API key for calling gemini-3.5-flash.

Basic flow:

Open Google AI Studio.
Find Get API key.
Select or create a project.
Create an API key.
Save the key to a local environment variable.

Python example:

import os
from google import genai

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

response = client.models.generate_content(
    model="gemini-3.5-flash",
    contents="Explain in three sentences what Gemini 3.5 Flash is best suited for."
)

print(response.text)

Node.js example:

import { GoogleGenAI } from "@google/genai";

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const response = await ai.models.generateContent({
  model: "gemini-3.5-flash",
  contents: "Explain in three sentences what Gemini 3.5 Flash is best suited for."
});

console.log(response.text);

curl example:

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash:generateContent" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"contents":[{"parts":[{"text":"Hello Gemini 3.5 Flash"}]}]}'

Public information suggests that the AI Studio free tier usually gives Gemini Flash models a daily request allowance. The exact numbers can vary by time, region, and account status. Common claims include around 1,500 requests per day, per-minute request limits, and token limits. Do not bake those numbers into a production plan; check Google’s current pricing and limits pages before launch.

Free Gemini 3.5 Flash Method 4: Gemini CLI

If you like the command line, look at Gemini CLI. It is useful for temporary scripts, repository summaries, file reading, and quick Q&A in the terminal.

Installation is usually:

`1`	`npm install -g @google/gemini-cli`

Then run:

gemini

The CLI is better suited to personal developer workflows than production integration. Production should still use API keys, service accounts, permission controls, and auditable calling patterns.

Free or Low-Friction Gemini Omni Access: Gemini App and Google Flow

Gemini Omni is a multimodal model for video creation and editing. Its core capability is not ordinary text Q&A, but multi-turn video editing with natural language while referencing image, text, video, and audio inputs.

The Google DeepMind page lists these entry points:

Gemini app.
Google Flow.
YouTube Shorts.

The page also notes that a Google AI subscription is required, and that features vary by subscription tier and region. So “free access” to Gemini Omni should be understood more carefully: some entry points may let free users see or try part of the experience, but full video editing may require a subscription, regional availability, or product rollout access.

If you only want to try it, use this order:

Open Gemini app first and check whether Gemini Omni or a related video editing entry is available.
Then open Google Flow: https://flow.google/
If you make short-form content, watch for Omni-related editing features in YouTube Shorts.

If the entry point is not visible, it usually does not mean you did something wrong. Your account, region, subscription tier, or rollout group may simply not qualify yet.

How Gemini Omni Is Best Used

Gemini Omni is more suitable for creators than for ordinary chat.

You can try these directions:

Upload or select a video and ask it to change the style.
Make a specific action in the video more exaggerated.
Use a reference image to replace an object or character in the scene.
Modify camera, action, environment, and style over multiple turns.
Combine sketches, reference images, audio, or video into a new output.

You can write prompts as if giving instructions to an editor:

Keep the original person and room structure unchanged. Change the effect after touching the mirror into liquid ripples. The motion should feel natural, and the lighting should not change abruptly.

For multi-turn editing, do not pack too many requirements into one request. A safer approach is:

Change the main action first.
Then change the style.
Then adjust the camera angle.
Finally tune sound, text, and rhythm.

This makes consistency easier to maintain and helps you identify which step caused a problem.

Common Pitfalls When Using Free Access

First, free quota is not production quota. A free API key is suitable for testing, personal tools, and prototypes, not for promising a stable service.

Second, do not send sensitive data to free or third-party entry points. This includes private code, customer data, contracts, keys, financial spreadsheets, and internal documents.

Third, check data-use settings. Free tiers may have different data-use policies, so review the settings in AI Studio or your Google account before use.

Fourth, video capabilities are usually more restricted than text capabilities. Gemini Omni-style video editing may be limited by subscription, region, queueing, duration, resolution, and content safety policies.

Fifth, be careful with third-party “unlimited free API” services. Many gateways rate-limit, forward requests, keep logs, or use opaque payment methods. Sensitive work should not go through these entry points.

Which Entry Point Should You Choose?

If you are an ordinary user:

Gemini 3.5 Flash: use Gemini app.
Gemini Omni: check Gemini app first, then Google Flow.

If you are a creator:

Use Google Flow to try Omni video workflows.
Use Gemini app for scripts, storyboards, prompts, and material descriptions.

If you are a developer:

Use AI Studio to debug prompts.
Use an API key to integrate gemini-3.5-flash.
Use Gemini CLI for personal terminal workflows.
For production, consider Vertex AI or the paid API.

If you are an enterprise:

Do not rely on free quotas.
Focus on permissions, logs, audits, data residency, compliance, and key management.
For video generation and editing, add watermarking, content review, and copyright processes.

Summary

Gemini 3.5 Flash has relatively clear free access paths: Gemini app, Google AI Studio, AI Studio API key, and Gemini CLI can all serve as low-friction entry points. It is suitable for chat, writing, coding, agent prototypes, and multimodal testing.

Gemini Omni focuses on video editing and multimodal creation. Its main entry points are Gemini app, Google Flow, and YouTube Shorts, but full capabilities are more likely to depend on subscription and region. It is best for creators to start with trials and concept validation, not to plan around it as a stable production service from day one.

The safest strategy is: test text and code tasks first with the Gemini 3.5 Flash free tier; validate video creation effects with Gemini Omni in Gemini app or Flow; when you need to launch something real, move to a formal setup with auditability, billing, and controlled permissions.

References:

What Is Gemini Omni? A Complete Look at Google's AI Video Multi-Turn Editing Model

Wed, 20 May 2026 23:11:58 +0800

Google DeepMind has published a page for Gemini Omni. Its positioning is direct: create content from any input, with the current focus starting from video.

If Nano Banana is more about image generation and editing, Gemini Omni feels more like a multimodal editing model for video. Users can modify a video step by step with natural language, with each later change building on the previous one, while trying to keep scenes, people, actions, and visual logic consistent.

Project page: https://deepmind.google/models/gemini-omni/

The Core Problem It Tries to Solve

Traditional video editing often requires timelines, layers, masks, keyframes, color grading, audio tracks, and a lot of manual work. AI video generation tools can already create clips from prompts, but they often run into two problems:

A generated result is hard to refine precisely.
During multi-turn edits, characters, scenes, styles, and actions can drift.

Gemini Omni is aimed at the second step: not just generating a video, but letting users keep asking for changes as if they were talking to an editor.

The project page describes it as a way to edit any video through natural, step-by-step conversation. Each edit builds on the prior result, with the goal of maintaining a coherent and unified scene.

Main Capabilities

Gemini Omni’s capabilities can be grouped into several areas.

The first is natural-language video editing. Users can directly ask the model to change a video’s aesthetic style, motion, or effects. For example, it can make a mirror ripple like liquid, turn a person into line art, a felt toy, or a transparent holographic wireframe, or transform an entire environment into 3D voxel art.

The second is action reconstruction. It can change what happens in a video, such as enlarging a hand-formed hole, making a toy produce the corresponding animal sound, or making building lights react to music.

The third is editing real video based on reference images. Users can provide an image reference and ask the model to place a building, sun, aircraft, or other object into a real video scene.

The fourth is maintaining consistency across multi-turn edits. The page shows a continuous editing flow: moving a violinist into a reference-image environment, removing the violin, and then changing the shot to an over-the-shoulder angle. This is closer to an actual creative process than a one-shot prompt.

The fifth is multi-input reference. Gemini Omni can combine image, text, video, and audio inputs into one output, supporting tasks such as style transfer, motion transfer, character replacement, and sketch-to-video generation.

Why It Emphasizes World Knowledge

Google repeatedly emphasizes that Gemini Omni is not only about making visuals look realistic. It also uses Gemini’s world knowledge, physical intuition, history, science, and narrative logic.

That matters. If a video model only optimizes for visual quality, it can easily produce illogical motion, confused object relationships, or mismatches between text and image. Gemini Omni’s goal is for video to look right while also being more coherent in story, physics, and meaning.

Examples on the page include:

A marble rolling through a chain-reaction track.
A claymation explanation of protein folding.
A stop-motion style explanation of how the hippocampus works.
Letters appearing in sync with objects in the scene.
On-screen words appearing one by one to the rhythm.

These examples suggest that Gemini Omni is not just a short-video effects tool. It tries to combine knowledge expression, storytelling, and audiovisual generation.

How It Relates to Veo, Flow, and Nano Banana

In Google’s current product lineup, Gemini Omni looks like a layer for multimodal creation and editing.

Veo is more focused on the video generation model itself, emphasizing cinematic video and audio generation. Google Flow is an AI creative studio for creators, suitable for organizing shots, assets, and video projects. Nano Banana is more focused on image creation and detailed editing. Gemini Omni emphasizes multimodal editing from any input to a consistent output, especially multi-turn natural-language control for video.

A simple way to understand it:

To generate high-quality video, watch Veo.
To organize video projects in a creative workflow, watch Google Flow.
To edit images, watch Nano Banana.
To modify video conversationally while referencing images, text, video, and audio, watch Gemini Omni.

Access Points

The page lists these access points:

Gemini app.
Google Flow.
YouTube Shorts.

However, it also notes that a Google AI subscription is required, and availability depends on subscription tier and region. In other words, not every user in every region can immediately access the full feature set.

For creators, Google Flow may be the most important entry point because it is closer to a complete creative workspace. For general users, Gemini app and YouTube Shorts may be lower-friction ways to try it.

Safety and Content Labels

The Gemini Omni page specifically mentions safety work. Gemini Omni Flash was developed in collaboration with internal safety and responsibility teams, with automated evaluations, human evaluations, human red teaming, automated red teaming, and pre-launch ethics and safety reviews.

For content transparency, the page says content created or edited with Omni in Gemini app, Google Flow, or YouTube will include imperceptible SynthID digital watermarks and C2PA Content Credentials. Users can verify content in Gemini app, with expansion to Chrome and Search planned later.

This is especially important for video models. The more realistic video generation and editing becomes, the more important source labeling, abuse prevention, and verification tools become.

Who It Is For

Gemini Omni is suitable for several types of users:

Content creators who want to modify video quickly with natural language.
Design teams that need to combine sketches, reference images, audio, and video assets into a finished clip.
People making short videos, ad concepts, educational explainers, and product visual drafts.
Creators building AI video workflows in Google Flow.
Developers and researchers watching the boundaries of multimodal video editing.

But it is not ideal for every scenario. Serious commercial films, brand key visuals, film production, and product launch videos still require human review, copyright checks, fact-checking, and asset management. AI can clearly speed up concept generation and first-draft iteration, but it should not replace final review.

How to Read Gemini Omni

The significance of Gemini Omni is that it moves AI video from “one-shot generation” toward “conversational editing.” That is closer to real creative workflows than simply improving visual quality.

If it performs reliably in multi-turn editing, consistency, reference control, audio-video synchronization, and content labeling, the way people use AI video tools will change. Users will no longer only write one long prompt and hope for the best; they will revise scenes, actions, styles, and narratives step by step like directors, editors, and designers.

What still needs to be observed is actual availability, pricing, regional limits, video length, resolution, copyright policy, and commercial-use rules. For ordinary creators, the most practical question is whether Gemini Omni can reliably handle multi-turn video editing inside Google Flow and Gemini app.

References:

Google DeepMind: Gemini Omni

Gemini 3.5 Is Here: Flash Leads as Google Focuses on Agents and Long-Running Tasks

Wed, 20 May 2026 22:51:31 +0800

Google officially released the Gemini 3.5 series on May 20, 2026. The first model available is Gemini 3.5 Flash. Its positioning is not just chat, but agents, code generation, and long-running complex task execution.

The message is clear: Google wants Gemini 3.5 to answer questions, but also to plan, execute, check results, and keep work moving across multi-step workflows.

Gemini 3.5 Flash Comes First

Gemini 3.5 Flash is already available to several groups:

General users can try it in the Gemini app and AI Mode in Google Search.
Developers can use it through Google Antigravity, Google AI Studio, and the Gemini API in Android Studio.
Enterprise users can access it through Gemini Enterprise Agent Platform and Gemini Enterprise.

Google also said Gemini 3.5 Pro is still in development, already being used internally at Google, and expected to launch next month.

This means the 3.5 series will continue the Flash and Pro split: Flash emphasizes speed, cost, and scalable execution, while Pro will likely target more complex and higher-capability use cases.

The Focus Is Agents and Coding

Google describes Gemini 3.5 Flash as one of its strongest models for agents and coding. The announcement says it beats some Gemini 3.1 Pro results on coding and agent benchmarks such as Terminal-Bench 2.1, GDPval-AA, MCP Atlas, and CharXiv Reasoning.

Most users do not need to care about every benchmark number. The more important point is that Google is pushing model capability toward executable workflows: not only writing code, but also migrating old projects, developing complex apps, organizing financial reports, analyzing data, and running repeated tests.

In the Antigravity development framework, Gemini 3.5 Flash can use multiple collaborating subagents to handle large tasks. Google showed examples such as reading the AlphaZero paper and building a playable game, converting legacy code to Next.js, and generating cityscapes and UI options in parallel.

The direction is clear: AI coding tools are moving from “generate a piece of code” toward “coordinate multiple agents to complete a project.”

Stronger Multimodal UI and Graphics

Gemini 3.5 Flash builds on Gemini 3’s multimodal foundation. Google says it can generate richer web UIs, interactive animations, and visual content.

The announcement includes examples such as:

Creating interactive animations for research papers.
Turning text descriptions into interactive hardware models.
Generating a complete brand concept for a school fundraiser.
Producing multiple UX options for a checkout flow in a short time.

This matters for developers and product teams. The model is no longer only writing explanations. It can participate in frontend prototypes, interaction design, and visualization work.

Enterprise Use: Automating Time-Consuming Workflows

Google listed several partner examples. Shopify uses subagents to analyze complex data and forecast merchant growth. Macquarie Bank is testing 3.5 Flash on documents over 100 pages to accelerate account-opening workflows. Salesforce is integrating it into Agentforce. Ramp uses it to improve OCR for complex invoices. Xero uses AI agents for administrative workflows. Databricks uses automated workflows to monitor data anomalies and suggest fixes.

These examples point to the same trend: enterprise adoption of large models is moving from one-off Q&A to workflow automation. Whether a model is inexpensive, fast, and stable over long tasks can matter more than whether one answer looks impressive.

Gemini Spark: A Personal AI Agent

Google also announced Gemini Spark, a personal AI agent powered by Gemini 3.5 Flash. Its goal is to run over long periods and proactively perform tasks under user guidance.

Gemini Spark has started rolling out to trusted testers. Google plans to open a beta next week to Google AI Ultra subscribers in the United States.

This is worth watching. Google Search, the Gemini app, Android, Workspace, and browser-related ecosystems already touch many parts of personal digital life. If a personal agent can connect with these entry points, its impact may be larger than a standalone chatbot.

Safety Moves Further Upstream

Google says Gemini 3.5 was developed under its Frontier Safety Framework, with strengthened protections for information security and CBRN-related risks. The announcement also mentions interpretability tools that help examine and understand model reasoning before responses are delivered.

This shows that frontier model releases are no longer only a capability race. The more a model emphasizes agents, autonomous execution, and long-running tasks, the more important safety controls, false refusal rates, harmful-output prevention, and interpretability become.

How to View Gemini 3.5

Gemini 3.5 Flash is not just another model launch. It looks more like Google’s bet on the next shape of AI products: models that can call tools, split tasks, coordinate execution, generate UIs, and enter personal and enterprise workflows.

For developers, the important things to watch are the real experience in Google Antigravity, AI Studio, the Gemini API, and Android Studio. For enterprises, the question is whether it can reliably reduce manual work in real workflows, not just score well on benchmarks.

Gemini 3.5 Pro is not publicly available yet. Once Pro ships, the differences between Flash and Pro in capability, price, speed, and context handling will decide which production scenarios each model fits best.

References:

Google Blog: Gemini 3.5

Google Gemini Spark Leak: A 24/7 Gemini Agent May Be Coming

Sun, 17 May 2026 11:58:08 +0800

Google has not officially released Gemini Spark.

Current information about it mainly comes from internal Gemini Web test screens, community screenshots, TestingCatalog reporting, and 36Kr / Xinzhiyuan’s summary of related leaks. The consistent picture is that Gemini Spark BETA may be an always-on AI Agent that Google is preparing. Its positioning is no longer just a chat assistant, but an “everyday AI agent” that can handle email, online tasks, and multi-step workflows in the background.

So the boundary should be clear first: this is a leak analysis, not an official Google announcement. All features, naming, and launch timing still need to be confirmed by Google.

Bottom line

Based on currently exposed information, Gemini Spark has three key points:

It may be a 24-hour online Agent inside the Gemini system, not a normal chat model.
It may use broader personal context, including Google apps, chat history, tasks, logged-in websites, and location.
Its risks are as large as its appeal, because it may involve information sharing, remote browser data, purchases, and third-party service calls.

If Google really launches Spark, Gemini’s role will change from “AI that answers questions” to “AI that continuously handles tasks for you.”

What Gemini Spark is

TestingCatalog reported on May 14, 2026 that Google is testing Gemini Spark BETA inside Gemini Web. The exposed welcome text describes it as an everyday AI agent that can help users 24/7 with inbox, online tasks, and more multi-step work.

The 36Kr / Xinzhiyuan article also says that after Spark was uncovered, the outside world saw a “full-time Agent” direction: it can stay on standby all day, process inboxes, execute online tasks, and may even involve purchases and information sharing.

This means Spark is not simply a new model name. It looks more like a Gemini product-layer upgrade: bringing Gemini out of the conversation window and into users’ email, web, calendar, tasks, and cross-app workflows.

How it may work

According to the hidden onboarding text disclosed by TestingCatalog, Gemini Spark may gather context from multiple sources, including:

Connected Apps.
skills.
chats.
tasks.
Websites the user has logged into.
Personal intelligence.
location.

This information would help Spark understand what the user wants to complete and call the necessary context while executing tasks. The text also says that, to complete some actions, Gemini may share necessary information with third parties, such as names, contact details, files, preferences, and information the user may consider sensitive.

If these descriptions prove accurate, Spark will work more like a context-aware agent system than a one-shot Q&A assistant. It will not only look at the current prompt, but may combine long-term preferences, connected apps, browser state, and task history.

Why it matters

The key to Gemini Spark is not one more chat entry point. It is that Google has a natural ecosystem entry point.

OpenAI and Anthropic can build powerful Agents, but they do not naturally own the full chain of Gmail, Calendar, Drive, Chrome, Android, and Workspace. If Google connects Spark into these products, users will not need to build many extra workflows before letting an Agent enter daily work.

This may bring three changes.

First, Gemini may move from passive Q&A to active execution. Users may no longer only ask “summarize this email”; they may ask it to continuously organize the inbox, track tasks, and take follow-up actions.

Second, Agents will rely more on personal context. The more it understands your email, calendar, files, browser state, and preferences, the more useful the result may be.

Third, permission boundaries will become more sensitive. Doing more also means users need to know more clearly when it can act, how far it can go, and whether confirmation is required.

Where the risks are

Several details in the onboarding text disclosed by TestingCatalog are worth watching.

First, Spark is experimental. Even if it launches, it should not be treated as a fully mature system that needs no supervision.

Second, although the system is designed to ask for permission before sensitive operations, the text also warns that it may share information or complete purchases without asking.

Third, to maintain session continuity, Gemini will save remote browser data, such as login details and remote code execution data. Users can clear these data in Settings and can also disable Connected Apps and Personal intelligence.

Taken together, these points show that Spark’s product direction is aggressive: it wants to be an Agent that can truly execute tasks, not only generate suggestions. But the closer it gets to real execution, the more it needs strict permissioning, auditing, confirmation, and rollback mechanisms.

Relationship with Remy and AI Ultra

TestingCatalog says Spark may be a renamed version of the agentic Gemini upgrade previously codenamed Remy, and may also relate to the Gemini Agent direction for Google AI Ultra subscribers.

If this clue is correct, Spark may not be a brand-new project from nowhere. It may be Google repackaging previously higher-end or more closed Agent capabilities and preparing to bring them to a wider audience.

36Kr / Xinzhiyuan also describes it as an upgrade from “Remy” to “Spark”: Gemini Agent is no longer just a feature, but is moving toward a 24/7 digital life manager.

But this is still a judgment based on leaked information. Whether Google will use Spark as the official name, whether it will be limited to AI Ultra, and whether a lighter subscription tier will appear all need official confirmation.

MCP, skills, and the tool ecosystem

The same batch of community screenshots also showed model selector entries such as MCP Tool Testing. The 36Kr article suggests this may hint that the new Gemini will natively support MCP third-party tool integration, with Thinking mode also being rebuilt.

This clue becomes more interesting when viewed together with Spark.

If Spark were only a chat assistant, skills and MCP would matter less. But if Spark is a long-running Agent, it needs to reliably call tools, access web pages, execute tasks, read and write context, and deliver results to users.

In other words, Spark may not be a single feature. It may be part of Google’s Agent tool ecosystem: the model handles understanding and planning, while skills / MCP / connected apps handle execution and expansion.

What it means for ordinary users

If Gemini Spark really launches, ordinary users may see these direct changes:

Email is not only summarized, but can be categorized, followed up, and turned into tasks.
Web tasks are not only suggested, but may be continuously executed in a remote browser.
Calendar, location, preferences, and previous chats become long-term Agent context.
Purchases, bookings, form filling, and similar actions may enter the AI execution range.

This sounds convenient, but users will need new habits: not only checking what the AI says, but also what it is preparing to do, what it has already done, whether it can be undone, and whether there is a record.

Future AI Agent experience may depend not only on model intelligence, but also on clear permission prompts, inspectable task logs, and recovery from mistakes.

What it means for developers and teams

For developers, Spark matters because Google may be moving Agents from “demo products” toward real workflow platforms.

If Spark can reliably connect Google apps, third-party tools, and browser state, developers will care about:

Whether APIs or extension mechanisms are open.
Whether MCP or skills can be connected by third parties.
Whether enterprise admins can control permissions, data retention, and audits.
Whether Agent execution failures have traceable logs.
Whether sandboxing, approval flows, and sensitive-operation confirmation are supported.

For teams, Spark may first enter high-frequency scenarios such as Gmail, Calendar, Docs, Drive, and Chrome. It may not be suitable for fully automated high-risk work at the beginning, but it is a strong fit for inbox triage, meeting follow-up, document organization, market research, and lightweight operations tasks.

How to read it now

This story is best understood as “high-confidence direction, low-certainty details.”

The high-confidence direction is that Google is pushing Gemini Agents to be more proactive, longer-running, and more deeply integrated with its ecosystem. The Gemini Web test text reported by TestingCatalog, community screenshots, and 36Kr’s summary of multiple leaks all point in the same direction.

The low-certainty details are the official name, launch timing, permission rules, subscription tiers, supported regions, API availability, and whether it will really be called Gemini Spark.

The safest view for now:

Do not treat Spark as an already released official product.
Treat it as a strong signal for Google’s next AI Agent direction.
Wait for official explanations around permissions, privacy, third-party data sharing, and remote browser data storage.

Summary

If Gemini Spark eventually launches, it may be a key step in Gemini’s move from chat assistant to always-on Agent. It is not just a model swap; it places Gemini into Google’s ecosystem of email, web, tasks, location, personal intelligence, and third-party services.

Its potential is large: more proactive, closer to real workflows, and easier to distribute to many users through Google’s ecosystem. Its risks are just as large: once AI can share information, save browser state, make purchases, and call third-party services, permission boundaries must be extremely clear.

So the most important question about Gemini Spark is not “how smart is it”, but how Google plans to make a 24-hour online AI Agent controllable, auditable, and trustworthy.

References:

Gemini 3.5 Pro Leak: Codenamed Cappuccino, Google Tries to Regain Momentum in Coding and Agents

Sun, 17 May 2026 11:47:27 +0800

Google has not officially released Gemini 3.5 Pro.

What we can see so far mainly comes from developer community screenshots, anonymous benchmarks, leakers, and media reports. On May 15, 2026, 36Kr / Xinzhiyuan reported that a next-generation Gemini checkpoint may be internally codenamed Cappuccino, and that related models have already surfaced in communities and benchmark platforms.

This information should not be treated as an official launch, but it points in a clear direction: Google is trying to address two gaps at once, coding and reasoning on one side, and always-on AI agents on the other.

Bottom line

This leak can be read in three layers:

Gemini 3.5 Pro has not been officially released, and Cappuccino looks more like an internal checkpoint or candidate build.
The leaked information suggests the new Gemini is improving in code generation, SVG / interactive web generation, and multimodal output.
Google’s parallel test of Gemini Spark may matter more than the model itself, because it points to a 24-hour personal AI agent.

In other words, this is not just a “model benchmark” story. It looks more like a product roadmap signal ahead of Google I/O: the model needs to catch up with GPT-5.5, while the agent layer needs to capture user workflows.

What Cappuccino is

The 36Kr article says a post from Lentils indicates that the Gemini 3.5 Pro checkpoint codenamed Cappuccino has started to appear. The community had been discussing Gemini 3.2 only hours earlier, but the latest leak jumped directly to 3.5.

If that naming is ultimately accurate, Google may want to frame the next Gemini as a larger version jump rather than a routine point release.

For now, Cappuccino should still be treated as a leaked internal codename. It does not mean Google has publicly launched the final model, and it does not guarantee that the final release name will be Gemini 3.5 Pro.

Why coding is the focus

The most discussed part of the leak is the new Gemini’s coding ability.

According to community screenshots and benchmark claims cited by 36Kr, the new model appears stronger at:

Generating SVG and visual components.
Generating interactive web apps.
Handling animation, 3D, adjustable control panels, and other complex frontend outputs.
Improving logical reasoning and code generation.

The article also cites Abacus.AI CEO Bindu Reddy as saying that 3.2 Flash is close to GPT-5.5 in coding and reasoning while being much cheaper. Other media sources reportedly believe the new Gemini roughly reaches the GPT-5.5 tier overall, but may not represent a qualitative leap.

That is why the phrase “matches GPT-5.5” needs caution. It is more of a relative judgment from different leaks and anonymous tests than an official Google benchmark result.

Why Google needs to catch up in coding

AI coding has moved from developer tooling into the center of foundation model competition.

OpenAI has Codex, and Anthropic has Claude Code. They serve engineers, but they also bring product managers, designers, and operators into workflows where natural language can produce runnable products.

By comparison, Google has Gemini and Antigravity, but it has not formed the same default entry point in developer mindshare. The 36Kr article also notes that Antigravity has not truly broken through externally, and that pricing, quota reminders, and experience stability have drawn community discussion.

So if the new Gemini needs to prove itself, coding is the most direct battlefield. The question is not only whether it can write code, but whether it can reliably produce complete interfaces, understand complex requirements, call tools, fix errors, and fit into real development workflows.

Spark may matter more than 3.5 Pro

In the same wave of leaks, Gemini Spark BETA also surfaced.

According to TestingCatalog and other sources, Spark is positioned like an always-on AI agent: it can process inboxes, execute online tasks, manage multi-step workflows, and connect context from Google apps, skill modules, chat history, scheduled tasks, logged-in websites, and location data.

That means Spark is not a normal chat entry point. It may be a system that stays online, continuously reads context, and performs tasks for users.

Its appeal is obvious: if Google can connect Gmail, Calendar, Chrome, Android, Workspace, and Gemini, Spark will have a distribution advantage that OpenAI and Anthropic cannot easily copy.

The risk is just as obvious. The 36Kr article mentions wording around Spark saying it may share information or complete purchases without asking. Even if the system is designed to request permission before sensitive operations, this kind of agent still raises privacy, authorization-boundary, and accidental-action risks.

What this means for ordinary users

If you are a regular Gemini user, the most important part of this leak is not the model name. It is three shifts.

First, Google may continue to strengthen the ability to produce complete results. Users have often complained that Gemini can be lazy with visual generation, SVG, and frontend pages. If the new model can produce several complete options in one pass, the experience will improve noticeably.

Second, coding ability may continue to move into lighter models. The leak repeatedly mentions Flash improvements in coding, reasoning, and interactive generation, which means complex tasks may not always require Pro models in the future.

Third, agents will become more proactive. If Spark launches, Gemini may no longer just answer questions. It may start taking over email, web tasks, purchases, calendars, and cross-app workflows over longer periods.

That is good for efficiency, but it creates a new challenge for permission management.

What this means for developers

Developers should watch two issues more closely.

The first is tooling. The 36Kr article says community screenshots showed an unreleased entry called MCP Tool Testing in the model selector. If Gemini natively supports MCP or third-party tool testing, it will be easier to connect it to developers’ own toolchains.

The second is cost and stability. Even if the new Gemini matches GPT-5.5 on some benchmarks, developers will ultimately judge three things: actual code quality, context stability, and whether pricing and quotas are predictable.

The past year of AI coding tool competition has shown that model capability is only the ticket in. What keeps developers is whether the tool can reliably edit code, run tests, read context, and handle edge cases in daily projects.

How to read this news now

This story is best understood as “strong signal, weak confirmation.”

The strong signal is that multiple community clues point to Google preparing a stronger new Gemini and a more proactive Gemini Spark Agent.

The weak confirmation is that Gemini 3.5 Pro has not been officially released, Cappuccino remains a leaked codename, and claims that it “matches GPT-5.5” still need validation through official Google benchmarks, third-party tests, and real user experience.

The safest view for now:

Do not treat it as a released product.
Treat it as an early preview of Google’s next Gemini direction.
Watch whether I/O or later official events confirm the model name, API availability, pricing, context window, tool calling, and agent permission boundaries.

Summary

The exposure of Gemini 3.5 Pro / Cappuccino suggests Google may be preparing a stronger next-generation Gemini push. It is not trying to fix one isolated capability, but a whole AI workflow: the model needs to write code better, generate interfaces, and handle complex reasoning, while Spark pushes Gemini toward an always-on agent.

But before an official release, all benchmarks and screenshots remain clues. What will decide whether Gemini 3.5 Pro can regain momentum is not whether the codename sounds good, but whether it can reliably win in real development, real office work, and real multi-step tasks.

References:

Gemini Intelligence on Android: Google Is Turning the Phone into a Proactive AI System

Sun, 17 May 2026 09:13:32 +0800

On May 12, 2026, Google published “A smarter, more proactive Android with Gemini Intelligence,” introducing Gemini Intelligence on Android. This is not a standalone chat app. It brings Gemini capabilities into Android, Chrome, Gboard, Autofill, widgets, and multi-device experiences, moving the phone from “wait for the user to tap” toward “proactively help the user complete tasks.”

In short, Google wants Android to move from an operating system toward an intelligence system. The phone no longer just opens apps, shows notifications, and runs settings. It can understand the screen, apps, voice, and personal context, then complete more complex actions with user confirmation.

Short Version

Gemini Intelligence on Android focuses on five areas:

Multi-step automation: Gemini can complete flows across apps, such as rides, shopping, and research.
Smarter Chrome browsing: summarize pages, compare information, and handle some repetitive web tasks on Android.
Upgraded Autofill: use Gemini and personal context to fill more complex forms.
Rambler: turn natural speech into clearer, more polished text.
Natural-language widgets: describe what you want, and Android generates custom widgets.

These features will start rolling out in summer 2026, first on select Samsung Galaxy and Google Pixel phones, and later to more Android devices including watches, cars, glasses, and laptops.

Multi-Step Automation: From Suggestions to Execution

The most important direction is letting Gemini complete multi-step tasks across apps.

Google gives examples such as booking a spin class, finding a course syllabus in Gmail and adding required books to a shopping cart, or seeing a travel poster and asking Gemini to find a similar trip on Expedia.

The hard part is not just understanding one sentence. The system needs to understand:

What is on the user’s current screen or image.
App information the user has authorized.
Which app should be opened next.
Which steps can be automated.
Which steps must pause for user confirmation.

Google emphasizes that Gemini acts on user instructions and stops when the task is done, with final confirmation remaining under user control. This is not a fully autonomous agent, but a mobile agent with human confirmation in the loop.

Screen and Image Context Matter More

One important change is screen context and image context.

Older phone assistants mostly relied on voice commands and fixed app integrations. Gemini Intelligence puts more emphasis on “seeing” the current screen. For example, if a user has a shopping list in notes, they can long-press the power button to summon Gemini and ask it to create a delivery cart from the list.

This means Android AI is not just a chatbot. It is trying to understand the user’s current operating environment. Future mobile AI competition may depend not only on who has the better model answer, but also on:

Whether the AI can understand the current screen.
Whether it can act across apps.
Whether it can track task progress in the background.
Whether it can reliably ask for confirmation at key points.

That is a major difference between mobile AI and web chat AI.

Chrome: From Search to Web Task Agent

Google says Android devices will get a smarter Gemini in Chrome starting in late June 2026.

It can help users research, summarize, and compare web content, and Chrome auto browse can handle some repetitive web tasks such as appointments and parking reservations.

This means Gemini in Chrome is not just a page-summary feature. It is moving toward a browser agent. The browser is already where users complete many web tasks. If Gemini can understand pages, fill information, compare options, and execute some steps, Chrome becomes a task execution surface.

The challenge is practical:

Websites are complex, so automated actions can fail.
Forms, payments, logins, and CAPTCHAs require caution.
Users need to know what Gemini did.
Final submission, payment, or booking should usually remain human-confirmed.

The hard part is not only model capability, but browser automation, safety boundaries, and user trust.

Autofill: From Password Filling to Complex Forms

Autofill with Google was mostly about passwords, addresses, and payment details. Google now wants to upgrade it into a smarter form assistant.

With Gemini’s Personal Intelligence, Android can use relevant information from connected apps to fill more complex form fields, including forms in Chrome.

This is very practical. Filling complex forms on mobile is painful: small screen, many fields, and information scattered across email, calendar, chats, and documents. If Gemini can organize and fill this information with user permission, it can save a lot of time.

Google also stresses that connecting Gemini and Autofill with Google is strictly opt-in. Users choose whether to connect them and can turn the connection on or off in settings.

That matters because Autofill touches personal details, addresses, accounts, payments, work information, and sensitive forms. The more useful it becomes, the more important explicit permission and easy opt-out become.

Rambler: Turning Speech into Sendable Text

Rambler is one of the more interesting new features.

Gboard already supports speech-to-text, but natural speech often includes repetition, pauses, filler words, and self-corrections. Rambler’s goal is to turn natural speech into clearer text that is ready to send.

It is useful when:

You want to dictate a message quickly without editing every word.
Your speech includes pauses, repetition, or filler.
You need to turn a rough thought into a more professional text, email, or chat message.
You switch between languages and want the system to understand context.

Google says Rambler will clearly show when it is enabled, and audio is used only for real-time transcription and not saved. This is a response to privacy and transparency concerns.

From a product perspective, Rambler upgrades “voice input” into “voice writing.” It does not only record what you said; it helps turn speech into sendable text.

Natural-Language Widgets

Gemini Intelligence also introduces Create My Widget. Users can describe a widget in natural language, such as “recommend three high-protein meal prep recipes every week,” and Android generates a custom widget for the home screen.

This points toward generative UI. Users no longer pick only from fixed widget templates; they describe the information and presentation they want.

If the idea matures, the phone home screen could become much more personal. Weather, schedule, health, commute, food, learning, and work reminders could all become dynamic modules generated around user needs.

But generative UI also needs stability. A widget is not a one-off chat response. It sits on the home screen for a long time and must be reliable, readable, configurable, and visually controlled.

Material 3 Expressive and Intelligent UI

Google also says Gemini Intelligence will bring design updates based on Material 3 Expressive.

This is not just decoration. When AI starts acting proactively, the UI needs to clearly show:

What the AI is doing.
Which steps are done.
Where user confirmation is needed.
How the user can cancel or change the action.

Proactive AI without clear UI easily makes users feel out of control. Design language becomes part of the AI product experience.

Availability and Rollout

According to Google, Gemini Intelligence features will start on the latest Samsung Galaxy and Google Pixel phones in summer 2026, then expand to more Android devices, including watches, cars, glasses, and laptops.

This is not a global all-at-once launch. Availability may depend on device, region, language, app support, and account settings.

If you want to try it, the realistic expectations are:

Watch Pixel and Samsung flagship phones first.
Watch for system updates after summer 2026.
Look for new toggles in Gemini, Chrome, Gboard, Autofill, and Android settings.
Not every region and language will support every feature at the same time.

What This Means for Android

Gemini Intelligence on Android is not just a bundle of small AI features. It changes Android’s product direction.

Traditional phone operating systems manage apps, notifications, permissions, files, and hardware. Google now wants the system to understand user intent and complete tasks across apps. If this works, Android’s competition will shift from “system features and app ecosystem” toward “how well the system can proactively help users do things.”

It also changes mobile AI competition:

Apple will emphasize on-device integration, privacy, and system-level control.
Google will emphasize Gemini, Search, Chrome, Android, and multi-device ecosystems.
Third-party AI apps will find it harder to compete with system-level entry points.
App developers will need to think about how their apps can be called by AI agents.

In the next few years, AI on phones may no longer be just a chat entry point. It may become the system-level execution layer.

Summary

Gemini Intelligence on Android is not about adding another Gemini chat box to the phone. It puts AI into Android’s operating flow. Multi-step automation, smarter Chrome browsing, Autofill, Rambler, and natural-language widgets all aim to turn the phone from a passive tool into a proactive assistant.

Whether it changes user habits depends on reliability, clear privacy controls, smooth cross-app operation, and keeping users in final control. At least from this announcement, Google is defining the next stage of Android as a proactive AI system, not just a traditional mobile operating system.

Reference:

Google Blog: A smarter, more proactive Android with Gemini Intelligence

Gemini Embedding 2: Putting Text, Images, Video, and Audio in One Vector Space

Mon, 04 May 2026 06:01:10 +0800

Google Developers Blog introduced how to build with Gemini Embedding 2. The model is now generally available through the Gemini API and Gemini Enterprise Agent Platform. The important point is not simply that it is a new embedding model, but that it maps text, images, video, audio, and documents into the same semantic space.

This broadens what retrieval systems can handle. Many RAG pipelines previously had to convert images, video, or audio into text or metadata before indexing them separately. Gemini Embedding 2 can process multimodal inputs directly, making it easier for agents, search systems, and classifiers to work with real business materials.

Original article: Building with Gemini Embedding 2: Agentic multimodal RAG and beyond

Model Capabilities

Gemini Embedding 2 supports more than 100 languages. A single request can process:

Up to 8,192 text tokens
Up to 6 images
Up to 120 seconds of video
Up to 180 seconds of audio
Up to 6 pages of PDF

Its key idea is a unified semantic space. Developers can place content from different modalities into one vector representation system, then use the same retrieval, clustering, or reranking logic to process it.

For example, a text description and an image can be included in the same embedding request:

from google import genai
from google.genai import types

client = genai.Client()

with open('dog.png', 'rb') as f:
    image_bytes = f.read()
result = client.models.embed_content(
    model='gemini-embedding-2',
    contents=[
        "An image of a dog",
        types.Part.from_bytes(
            data=image_bytes,
            mime_type='image/png',
        ),
    ]
)

print(result.embeddings)

If you want a separate embedding for each input rather than one aggregated vector, you can use the Batch API. The original article also notes that Agent Platform support for this kind of batch workflow is still in progress.

What It Means for RAG

Multimodal embeddings are useful for agentic RAG. An AI agent may need to inspect a code repository, PDFs, screenshots, charts, audio meeting notes, and product images at the same time. If all of these materials can enter the same semantic space, the retrieval pipeline no longer needs a separate entry point for every format.

Google recommends using task prefixes according to the goal of the task, so the embeddings better match the retrieval objective. For example, question answering, fact checking, code retrieval, and search results can use different prefixes:

# Generate embedding for your task's query:
def prepare_query(query):
    return f"task: question answering | query: {content}"
    # return f"task: fact checking | query: {content}"
    # return f"task: code retrieval | query: {content}"
    # return f"task: search result | query: {content}"
# Generate embedding for document of an asymmetric retrieval task:
def prepare_document(content, title=None):
    if title is None:
        title = "none"
    return f"title: {title} | text: {content}"

This kind of prefix is suitable for asymmetric retrieval: user queries are often short, while documents are often long. Formatting query and document differently for the task can improve matching between short queries and long documents.

The original article gives two real-world examples:

Harvey saw a 3% increase in Recall@20 precision on legal retrieval benchmarks compared with its previous embeddings.
Supermemory saw a 40% increase in Recall@1 search accuracy and uses it across memory, indexing, search, and Q&A pipelines.

These numbers do not mean every scenario will improve by the same amount, but they show that multimodal embeddings are already producing results in real retrieval products, not only demos.

Visual Search

Gemini Embedding 2 is also suitable for image-to-image search, image-text hybrid search, and product identification. The original article mentions Nuuly, URBN’s clothing rental company, using it to match photos of untagged garments in warehouses against its catalog. Match@20 improved from 60% to nearly 87%, and the overall successful identification rate rose from 74% to over 90%.

The point in this type of scenario is not content generation, but understanding which inventory item, document, or product record is closest to a given image. If your business has many images, video clips, or scanned documents, multimodal embeddings can be more natural than text-only indexing.

Search Reranking

Embeddings can also be used for reranking. A common approach is to first retrieve a set of candidate results, then calculate the similarity between each candidate and the user’s query, pushing more relevant content to the top:

# 1. Define a function to calculate the dot product (cosine similarity)
def dot_product(a: np.ndarray, b: np.ndarray):
  return (np.array(a) @ np.array(b).T)
# 2. Retrieve your embeddings
# (Assuming 'summaries' is your list of search results)
search_res = get_embeddings(summaries)
embedded_query = get_embeddings([query])

# 3. Calculate similarity scores
sim_value = dot_product(search_res, embedded_query)

# 4. Select the most relevant result
best_match_index = np.argmax(sim_value)

The original article also mentions another idea: first ask the model to generate a baseline hypothetical answer from its internal knowledge, embed that answer, and compare it with candidate content to find the most semantically relevant result. This is especially useful for Q&A-style RAG.

Clustering, Classification, and Anomaly Detection

Beyond retrieval, embeddings are also useful for clustering, classification, and anomaly detection. Unlike the asymmetric question-answering retrieval above, these are symmetric tasks, where the same task prefix can be used for both query and document:

# Generate embedding for query & document of your task.
def prepare_query_and_document(content):
    # return f'task: clustering | query: {content}'
    # return f'task: sentence similarity | query: {content}'
    # return f'task: classification | query: {content}'

These tasks can be used for sentiment classification, content moderation, similar asset grouping, and anomaly discovery. They can also help agents organize large amounts of context before moving into later reasoning steps.

Storage and Cost

Gemini Embedding 2 outputs 3,072-dimensional vectors by default. It uses Matryoshka Representation Learning, so vectors can be truncated to smaller dimensions with output_dimensionality. Google recommends 1,536 or 768 dimensions when efficiency is the priority:

result = client.models.embed_content(
    model="gemini-embedding-2",
    contents="What is the meaning of life?",
    config={"output_dimensionality": 768}
)

Vectors can be stored in Agent Platform Vector Search, Pinecone, Weaviate, Qdrant, ChromaDB, and similar systems. For cost, the original article notes that the Batch API provides higher throughput and reaches 50% of the default embedding price.

How Developers Can Use It

If you already have text-based RAG, you can start with two incremental upgrades:

Put PDFs, screenshots, image descriptions, and text documents into the same index, then test whether retrieval recall becomes more stable.
Add task prefixes for different tasks, such as question answering, fact checking, code retrieval, and product search. Do not process all content with the same embedding format.

If you are building a new product, consider these directions first:

Enterprise knowledge bases: retrieve documents, charts, presentation screenshots, and meeting materials together.
Visual search: use images, text, or mixed inputs to find products, assets, design drafts, and archives.
Agent toolchains: let coding agents, research agents, or customer support agents retrieve business materials in multiple formats.
Content governance: classify, cluster, and detect anomalies across text, images, and video clips.

The value of Gemini Embedding 2 is that it turns multimodal materials into one searchable asset system. For developers, this reduces the need for an intermediate “convert to text, then retrieve” layer and makes RAG systems closer to the shape of real-world data.

Gemini Comes to Cars with Google Built-in: In-Car Voice Assistants Start to Feel More Like Real AI Assistants

Fri, 01 May 2026 06:09:57 +0800

Google announced on April 30, 2026 that Gemini is starting to roll out to cars with Google built-in, as an upgraded version of Google Assistant.

The point is not simply that cars are getting another AI assistant. It is that in-car voice interaction is moving from fixed commands toward more natural, continuous conversation. Users no longer need to remember rigid command formats. They can speak more naturally and ask Gemini to help with navigation, messages, vehicle information, and even some in-car settings.

Starting with English users in the United States

According to Google, this update will cover both new and existing vehicles, as long as the vehicle supports Google built-in and the user is signed in to their Google account in the car.

The rollout will begin with English-language users in the United States, then expand to more languages and countries. Eligible users will see an option in the car to upgrade to Gemini. After upgrading, they can call up Gemini in several ways:

Say Hey Google
Tap the microphone on the home screen
Use the voice button on the steering wheel

This shows that Google is not turning Gemini into a new entry point that users have to learn from scratch. It keeps the existing in-car voice entry point, while replacing the underlying assistant with a stronger Gemini experience.

In-car voice no longer depends only on fixed commands

A common problem with traditional in-car voice assistants is that they can do quite a lot, but users have to speak in a very “standard” way. As soon as the request becomes a little complex, the assistant may fail to understand it or only perform a basic action.

With Gemini in the car, Google is emphasizing natural conversation. For example, a user can simply say:

I need to grab lunch, find some highly rated sit-down restaurants along the way. I’m not in a rush, oh, and I’d like to eat outside.

Gemini can use Google Maps information to find suitable restaurants along the route. The user can then follow up by asking about parking or vegetarian options, without starting a whole new search.

This interaction fits the driving context better. When driving, it is hard to repeatedly filter, tap, and revise options as you would on a phone. If a voice assistant can understand more complete intent, it can noticeably reduce distraction.

Maps, messages, and music become easier to handle

The examples Google gives are mostly built around the most common needs while driving.

The first category is route and place search.

Gemini can use Google Maps information to find restaurants, attractions, or charging stations along the way, and it can also answer questions related to the current route. For example, when passing near a stadium, the user can ask whether there is an event nearby and whether it will affect traffic.

The second category is message handling.

Users can ask Gemini to summarize new text messages and then reply based on the context. For example, they can ask it to tell a friend “I’m on my way” and include the estimated arrival time. If they want to change the message, they can add more instructions without starting over.

The third category is music and ambience.

Users do not necessarily need to know the name of a radio station or a specific playlist. They can simply describe what they want to hear. For example, they can ask for a jazz radio station, or ask YouTube Music to play upbeat ’70s folk-rock for a mountain drive while skipping slow ballads.

These functions are not entirely new by themselves. Gemini’s value is in handling multiple conditions in a single natural-language request, instead of forcing users back into fixed commands.

Gemini Live lets people keep talking while driving

Google also mentioned that Gemini Live will enter the in-car experience and is currently in beta. Users can tap the Gemini Live button or say Hey Google, let's talk to start a more free-flowing conversation.

This scenario is closer to learning and brainstorming while driving. For example, when driving to Lake Tahoe, users can ask Gemini to share local history and fun facts. If something sounds interesting, they can interrupt and ask follow-up questions. Gemini can also help plan hikes and activities after arrival.

The difference from traditional in-car assistants is clear. A traditional assistant is more like a tool button; Gemini Live is more like a voice interface that supports continuous conversation.

Owner’s manuals and real-time vehicle status are the key differences

More importantly, Gemini does not only answer general questions. Google says it has worked with automakers to integrate Gemini more deeply with vehicle systems.

This brings several capabilities closer to the car itself.

First, users can ask about vehicle features.

For example: “How should I prepare my car for an automatic car wash?” or “My garage ceiling is low and the trunk is hitting it. How do I program the trunk so it doesn’t open all the way?” Gemini can answer based on manufacturer-provided owner’s manuals, tailored to the specific vehicle model. The availability and level of detail will vary by brand and model.

Second, EV users can ask about real-time battery level and range.

For example, they can ask for the current battery level, the estimated battery level on arrival, or ask Gemini to find nearby charging stations. Gemini can also combine this with Google Maps and help find nearby cafes while charging.

Third, some in-car settings can be adjusted through natural language.

Google’s example is a user saying that the car is foggy and freezing. Gemini can understand the intent, turn up the heat, and switch on the defroster.

These capabilities are more practical than simply moving a chatbot into the dashboard. A car is an environment with clear state, hardware capabilities, and safety boundaries. If an AI assistant can understand vehicle context, its value is much higher than ordinary Q&A.

The boundaries of in-car AI matter even more

The requirements for AI in a car are different from those on a phone or web page.

When driving, users cannot keep looking at the screen or spend much attention correcting AI. The assistant needs to be concise, reliable, and avoid creating new burdens in critical situations.

So Gemini entering cars does not mean every complex task belongs in the car. A more reasonable direction is:

Reduce the operation cost of navigation and information lookup
Replace multi-level menus with natural language
Help users quickly understand vehicle features
Handle messages and media without increasing distraction
Give EV users smoother charging and route information

On the other hand, high-risk operations still need clear boundaries. Actions that affect driving safety, messages that require confirmation, and vehicle-control operations should all have sufficiently explicit confirmation flows.

Conclusion

Gemini coming to cars with Google built-in is another step in AI assistants expanding from phones and web pages into everyday environments.

Its significance is not that people can finally “chat” in the car. It is that in-car voice assistants are beginning to understand more complex intent and combine maps, messages, music, owner’s manuals, and some vehicle-status information to complete tasks.

If the rollout goes well, in-car voice interaction may gradually move from “remember the command” to “describe the need.” That matters for driving, because a genuinely good in-car AI should not require the driver to give it too much attention.

Reference link:

Your car with Google built-in is about to get smarter, thanks to Gemini - Google Blog

How to Use Google Nano Banana for Image Cutouts

Thu, 09 Apr 2026 20:10:48 +0800

This article uses a practical Python script to show how to call Google’s Nano Banana image editing capability for product-image cutouts.

The goal of this implementation is very clear:

Read product images from a directory
Call a Google image model to remove the background
Apply one more round of local transparent-background cleanup to the returned image
Export the final result as a transparent PNG

If you already have a batch of white-background product photos, headset images, or cable images and want to quickly generate transparent-background assets for e-commerce use, this approach is very direct.

What this code does

This script is mainly divided into 4 parts:

Define a prompt so the model understands it should remove the background, keep the subject intact, and avoid adding shadows
Call the google-genai image generation interface
Extract the image result from the model response
Use local logic to turn light-colored edge background areas transparent and reduce leftover halos

In other words, it does not simply send the image to the model and stop there. It combines model editing with local post-processing.

Before you run it

Install the dependencies first:

`1`	`.\.venv\Scripts\python.exe -m pip install google-genai pillow`

How to get GEMINI_API_KEY

GEMINI_API_KEY is the key used when calling the Gemini API. According to Google’s official quickstart, if you do not already have one, you can create it directly in Google AI Studio.

The process is straightforward:

Open Google AI Studio.
Sign in with your Google account.
Find the Get API key or API keys page.
Create a new API key.
Copy the generated key.
Configure it as a local environment variable for the script to read.

If there is no available project on the page yet, you typically need to finish project initialization first and then return to the API key page to create the key.

After you have the key, configure the environment variable:

`1`	`$env:GEMINI_API_KEY="your_api_key"`

If you are using cmd, you can write:

`1`	`set GEMINI_API_KEY=your_api_key`

If both GEMINI_API_KEY and GOOGLE_API_KEY are set, the runtime will usually prefer GOOGLE_API_KEY, so it is better to keep only one of them to avoid confusion.

Example directory structure

The script accepts two arguments:

input_dir: input image directory
output_dir: output image directory

For example:

images/
  product1.jpg
  product2.png

output/

How to run it

Assuming the script file is named cutout.py, run it like this:

`1`	`.\.venv\Scripts\python.exe .\cutout.py .\images .\output`

If you want to switch models, you can also pass it explicitly:

`1`	`.\.venv\Scripts\python.exe .\cutout.py .\images .\output --model gemini-2.5-flash-image`

The script will iterate over these file types in the input directory:

.jpg
.jpeg
.png
.webp

After processing, it will generate transparent-background PNG files with matching names in the output directory.

Core API call flow

The key code that actually calls Google Nano Banana is here:

response = client.models.generate_content(
    model=model,
    contents=[PROMPT, image],
)

Two pieces of content are passed in here:

A text prompt, PROMPT
A PIL.Image

The prompt asks the model to remove the full background from the product image, keep only the subject, and pay attention to a few important requirements:

Keep the full product intact
Preserve thin lines and cable details
Clean up inner holes and loop areas
Do not add new objects
Do not add shadows

Prompts like this have a big effect on cutout quality, especially for details such as earphone wires, transparent edges, and hollow regions.

Why local post-processing is still needed

After the model returns the result, the script does not save it directly. It also runs make_transparent_from_borders(image).

The idea behind this step is:

Start from the outer border of the image and find light-colored background pixels
Use breadth-first search to mark all connected light-colored regions
Convert those regions to transparent in one pass

The benefit is that it can further clean up leftover white edges, light gray backgrounds, and edge areas that are not clean enough.

The condition used to decide whether a pixel is background is here:

def is_light_background_pixel(r: int, g: int, b: int) -> bool:
    brightness = (r + g + b) / 3
    spread = max(r, g, b) - min(r, g, b)
    return brightness >= 170 and spread <= 35

In simple terms, this means:

The overall color must be bright enough
The RGB channel difference cannot be too large

This is especially suitable for product images with white, light gray, or near-solid-color backgrounds.

Full source code

The complete source code is preserved below so you can reuse or modify it directly:

from __future__ import annotations

import argparse
import os
from pathlib import Path
from collections import deque

from PIL import Image

try:
    from google import genai
except ImportError as exc:  # pragma: no cover
    raise SystemExit(
        "Missing dependency: google-genai. Install it with "
        r"'.\.venv\Scripts\python.exe -m pip install google-genai'."
    ) from exc


PROMPT = (
    "Remove the entire background from this product photo and return only the product "
    "on a fully transparent background as a PNG. Keep the full product intact, preserve "
    "thin cable details, clean the inner loops and holes, and do not add any new objects "
    "or shadows."
)


def is_light_background_pixel(r: int, g: int, b: int) -> bool:
    brightness = (r + g + b) / 3
    spread = max(r, g, b) - min(r, g, b)
    return brightness >= 170 and spread <= 35


def to_pil_image(image_obj) -> Image.Image:
    if isinstance(image_obj, Image.Image):
        return image_obj
    pil_image = getattr(image_obj, "_pil_image", None)
    if isinstance(pil_image, Image.Image):
        return pil_image
    as_pil = getattr(image_obj, "pil_image", None)
    if isinstance(as_pil, Image.Image):
        return as_pil
    raise TypeError(f"Unsupported image object type: {type(image_obj)!r}")


def make_transparent_from_borders(image: Image.Image) -> Image.Image:
    rgba = image.convert("RGBA")
    width, height = rgba.size
    pixels = rgba.load()

    visited: set[tuple[int, int]] = set()
    queue: deque[tuple[int, int]] = deque()

    def push_if_bg(x: int, y: int) -> None:
        if (x, y) in visited:
            return
        r, g, b, _ = pixels[x, y]
        if is_light_background_pixel(r, g, b):
            visited.add((x, y))
            queue.append((x, y))

    for x in range(width):
        push_if_bg(x, 0)
        push_if_bg(x, height - 1)
    for y in range(height):
        push_if_bg(0, y)
        push_if_bg(width - 1, y)

    while queue:
        x, y = queue.popleft()
        for nx, ny in ((x - 1, y), (x + 1, y), (x, y - 1), (x, y + 1)):
            if 0 <= nx < width and 0 <= ny < height:
                push_if_bg(nx, ny)

    for x, y in visited:
        pixels[x, y] = (0, 0, 0, 0)

    return rgba


def save_first_image_part(response, dst: Path) -> None:
    parts = getattr(response, "parts", None)
    if parts is None and getattr(response, "candidates", None):
        parts = response.candidates[0].content.parts

    if not parts:
        raise RuntimeError("Model returned no content parts.")

    for part in parts:
        inline_data = getattr(part, "inline_data", None)
        if inline_data is None and isinstance(part, dict):
            inline_data = part.get("inline_data")

        if inline_data is None:
            continue

        if hasattr(part, "as_image"):
            image = to_pil_image(part.as_image())
            dst.parent.mkdir(parents=True, exist_ok=True)
            make_transparent_from_borders(image).save(dst)
            return

        data = getattr(inline_data, "data", None)
        mime_type = getattr(inline_data, "mime_type", "")
        if data:
            dst.parent.mkdir(parents=True, exist_ok=True)
            with open(dst, "wb") as handle:
                handle.write(data)
            with Image.open(dst) as img:
                processed = make_transparent_from_borders(img)
                processed.save(dst.with_suffix(".png"))
            if dst.suffix.lower() != ".png":
                dst.unlink(missing_ok=True)
            return

    raise RuntimeError("Model returned text only and no edited image.")


def process_image(src: Path, dst: Path, client, model: str) -> None:
    with Image.open(src).convert("RGBA") as image:
        response = client.models.generate_content(
            model=model,
            contents=[PROMPT, image],
        )
    save_first_image_part(response, dst)


def main() -> None:
    parser = argparse.ArgumentParser(description="Use Nano Banana / Gemini image editing to cut out product images.")
    parser.add_argument("input_dir", type=Path)
    parser.add_argument("output_dir", type=Path)
    parser.add_argument("--model", default="gemini-2.5-flash-image")
    args = parser.parse_args()

    api_key = os.environ.get("GEMINI_API_KEY")
    if not api_key:
        raise SystemExit("Missing GEMINI_API_KEY environment variable.")

    client = genai.Client(api_key=api_key)
    exts = {".jpg", ".jpeg", ".png", ".webp"}

    for src in sorted(args.input_dir.iterdir()):
        if not src.is_file() or src.suffix.lower() not in exts:
            continue
        dst = args.output_dir / f"{src.stem}.png"
        process_image(src, dst, client, args.model)
        print(dst)


if __name__ == "__main__":
    main()

Good places to improve it further

If you plan to use this script for batch production, you can continue improving it in these directions:

Add retry logic so one failed image does not stop the whole batch
Add logs to make it easier to identify which image failed
Make background thresholds configurable
Support recursive scanning of subdirectories
Add a side-by-side preview of the original and processed result

Summary

If you only want the shortest explanation of how to use Google Nano Banana for cutouts, the core process is just three steps:

Install google-genai and Pillow
Set GEMINI_API_KEY
Pass the prompt and image to client.models.generate_content()

The value of this code is that it does more than just call the model. It also adds transparent-background post-processing, which makes it more suitable for direct product-image cutout work.

Google Gemma 4 Model Comparison: How to Choose Between 2B/4B/26B/31B

Sun, 05 Apr 2026 08:30:00 +0800

Gemma 4 focuses on multimodality and local offline inference, with a full range from lightweight to high-performance models. For most local deployment users, the key is not choosing the largest model, but choosing the one that best matches hardware and task needs.

Gemma 4 Model Comparison

The table below is for quick model selection. Actual performance and resource usage should be validated in your own environment.

Model	Parameter Size	Positioning	Key Strengths	Main Limitations	Recommended Scenarios
Gemma 4 2B	2B	Ultra-lightweight	Low latency, low resource usage, lowest deployment barrier	Limited performance on complex reasoning and long task chains	Mobile, IoT, lightweight Q&A, simple automation
Gemma 4 4B	4B	Lightweight enhanced	Stronger understanding and generation than 2B, still easy to deploy locally	Limited ceiling for heavy coding and complex agent tasks	Local assistant, basic document work, multilingual daily tasks
Gemma 4 26B	26B	High-performance (MoE)	Better reasoning and tool use, suitable for production workflows	Significantly higher VRAM requirement and hardware threshold	Coding assistant, complex workflows, enterprise internal agents
Gemma 4 31B	31B	High-performance (dense)	Best overall capability and stronger stability on complex tasks	Highest resource cost and tuning complexity	Advanced reasoning, complex coding tasks, heavy automation

How to Choose: Start from Hardware and Tasks

If your top concern is whether it runs smoothly, use this guideline:

8GB VRAM: prioritize 2B/4B.
12GB VRAM: prioritize 4B or quantized variants of larger models.
24GB VRAM: focus on 26B, and evaluate quantized 31B based on workload.
Higher VRAM or multi-GPU: consider high-precision 31B setups.

Prioritize stability and inference speed first, then scale up model size gradually.

Four Typical Use Cases

1) Local General Assistant

Preferred model: 4B
Why: strong balance between cost and quality, suitable for long-running local use.

2) Coding and Automation

Preferred model: 26B
Why: more stable in multi-step tasks, tool calls, and script generation.

3) Advanced Reasoning and Complex Agents

Preferred model: 31B
Why: stronger robustness under complex context.

4) Edge Devices and Lightweight Offline Use

Preferred model: 2B
Why: easiest to deploy on resource-constrained devices.

Deployment Suggestions (Ollama)

A practical approach is to iterate in small steps:

Start with 4B to establish a baseline (latency, memory, quality).
Build a fixed test set from real tasks (for example, 20 common questions + 10 automation tasks).
Compare 26B/31B against that set for accuracy, latency, and VRAM cost.
Upgrade only when the gain is clear.

This avoids jumping to a large model too early and running into lag, low throughput, and maintenance overhead.

Conclusion

The real value of Gemma 4 is not just larger parameter counts, but a practical model ladder from lightweight to high-performance:

For low-cost fast rollout: start with 2B/4B.
For production-grade local AI workflows: prioritize 26B.
For advanced reasoning and heavy automation: move to 31B.

In most cases, the best Gemma 4 choice is not the biggest model, but the one with the best fit for your hardware and task goals.