Gemini on KnightLi Blog

Gemini 3.5 Flash positioning and strengths: why it fits high-frequency, multimodal, low-latency use cases

Sun, 24 May 2026 08:43:24 +0800

The keywords for Gemini 3.5 Flash are not “the strongest,” but “high-frequency, fast, cost-efficient, and easy to integrate.” It is more like the workhorse model in the Gemini family: it may not be the model you use for the hardest reasoning tasks, but it is well suited for real production workloads such as Q&A, summarization, customer support, content processing, multimodal understanding, lightweight coding assistance, and automated workflows.

The key to understanding Flash is not to treat it as a replacement for a Pro-class flagship model. It is better understood as a model tier optimized for throughput and response speed. For developers and enterprises, the real cost of many AI applications is not only the strongest single response, but the latency, stability, price, and context-handling ability across thousands or millions of daily requests.

Product positioning

The Gemini family usually separates models into different tiers. Flagship models handle more complex reasoning, planning, and difficult tasks. Flash models emphasize speed, cost, and large-scale invocation.

The positioning of Gemini 3.5 Flash can be summarized as:

More suitable than Pro for high-frequency calls.
More capable than tiny lightweight models for complex input.
Optimized for low latency and high throughput.
Suitable for multimodal input and long-context processing.
Better as the default model inside applications, not only as a model for rare difficult requests.

This type of model is best for tasks that run many times every day. Its value is not just answer quality in one call, but whether it can reliably process large amounts of text, images, audio, video, or structured information at manageable cost.

Why Flash matters

When AI products move into production, a practical issue appears: the strongest model is useful, but not every request deserves the strongest model.

For example:

A user asks an ordinary customer-support question.
A system summarizes a meeting transcript.
A backend classifies a batch of tickets.
An app explains an uploaded image.
An automation extracts fields from an email.
An agent reads a set of documents before deciding the next step.

These tasks need models that are reliable, cheap, and fast, but they do not always require the full reasoning power of a flagship model. That is where Flash matters: it puts “strong enough” and “fast enough” in the same place.

If an AI application serves many users, the default model cannot be chosen only by peak capability. Average request cost, response speed, concurrency, and failure rate matter just as much. Flash is an application-layer model for that reality.

Advantage 1: low latency and high throughput

The most direct advantage of Flash is speed.

For chat products, retrieval-augmented search, support bots, real-time writing assistance, and agent workflows, latency directly affects user experience. Users may not know model parameters or benchmark results, but they immediately feel whether the product keeps them waiting.

Low latency brings several benefits:

Conversations feel more real-time.
Multi-step tool calls do not slow down as much.
Agents can make intermediate decisions more often.
Backend batch processing finishes faster.
Product teams can place AI features into more small workflows.

This matters especially for agent applications. A model does not answer only once; it repeatedly judges, calls tools, reads context, and generates the next action. Lower single-call latency improves the whole chain.

Advantage 2: better cost for scale

Another core value of Flash is cost.

When enterprises and developers put AI applications into production, they usually care about three questions:

How much does each call cost?
How many calls happen per day?
Are cost and latency controllable at peak concurrency?

If a task runs hundreds of thousands of times per day, even a small per-call price gap becomes large over time. Flash-style models are designed so that most requests do not have to go directly to the most expensive and heaviest model.

A common pattern is tiered routing:

Ordinary requests go to Flash by default.
Difficult problems, complex planning, and long-chain reasoning escalate to Pro.
Simple classification or fixed-format extraction can go to even lighter models.

This lets an AI system keep high-end capability while controlling everyday cost.

Advantage 3: multimodal input fits real applications

The Gemini family has long emphasized multimodal capability. Flash is valuable because it is not only for text requests; it can also handle images, audio, video, documents, and related inputs.

That matters in real products. Business data is often not pure text:

Users upload screenshots for support.
Customer support needs to understand a photo of a problem.
Education products process images of exercises.
Content platforms analyze video clips.
Office workflows read PDFs, spreadsheets, and presentations.
E-commerce products analyze product images and user descriptions.

If multimodal understanding depends only on expensive flagship models, many high-frequency scenarios are hard to scale. Flash brings multimodal understanding into a model tier better suited for large-scale invocation.

Advantage 4: long context makes it good at reading material

Long context is an important Gemini-family capability. For Flash, long context is not simply about stuffing everything into the prompt; it lets the model handle more information-organization tasks.

Examples include:

Summarizing long documents.
Reading product manuals.
Analyzing meeting notes.
Organizing multi-page PDFs.
Comparing contracts or proposals.
Providing agents with large task backgrounds.

Long context combined with lower cost is well suited for workflows that first read a lot of material and then produce actionable results. Flash does not need to solve extremely hard reasoning tasks every time. It can include more context in one pass, which is useful for office work, customer support, knowledge bases, and developer assistance.

Advantage 5: suitable as a default model

Many AI products need a “default model.” It does not have to be the most expensive or strongest, but it must satisfy several conditions:

Stable quality on most questions.
Fast response.
Manageable cost.
Ability to handle multimodal input.
Sufficient long-context support.
Easy API and product integration.

This is where Gemini 3.5 Flash has an advantage. It is suitable as the default entry point: handle most requests first, and route complex tasks to stronger models when needed.

This pattern will become increasingly common. Future AI systems will not simply “choose one model”; they will use Flash as the workhorse, Pro as the escalation path, and smaller models for edge tasks.

Suitable scenarios

Gemini 3.5 Flash is well suited for:

Customer-support Q&A and answers after knowledge-base retrieval.
Long-document summaries, report organization, and meeting notes.
Multimodal understanding of images, screenshots, PDFs, and video clips.
Real-time AI assistants inside apps.
Content moderation, classification, and tag generation.
Information extraction from emails, tickets, and forms.
Intermediate decisions and context compression in agent workflows.
Code explanation, lightweight fix suggestions, and documentation generation.
Education products for exercise explanation and study assistance.

These scenarios share the same traits: high request volume, sensitivity to user wait time, complex input types, and no need for flagship-level deep reasoning every time.

Where Flash should not be the only model

Flash is not universal. It is optimized for high-frequency and low-latency use, but that does not mean every problem should use only Flash.

The following scenarios still fit stronger Pro-class models better, or at least require tiered routing:

Complex mathematics and rigorous proofs.
Long-chain planning and multi-step strategic reasoning.
High-risk legal, medical, or financial judgment.
Deep refactoring plans for large codebases.
Complex agent tasks requiring high reliability.
Professional reports with extremely low tolerance for hallucination.

A safer strategy is to let Flash handle, judge, and organize first; when task complexity rises, escalate to a stronger model.

Relationship with Pro-class models

Flash and Pro should not be understood as “which one replaces the other.” They have different jobs.

Flash is the everyday workhorse:

Fast.
Cost-friendly.
Suitable for high concurrency.
Good for multimodal and long-context applications.
Suitable for default product flows.

Pro is the hard-task model:

Better for complex reasoning.
Better for difficult planning.
Better for high-value requests.
Better for small numbers of important deep-analysis tasks.

Good AI products usually combine the two instead of choosing only one.

How developers should use it

If you want to integrate Gemini 3.5 Flash into a product, consider these patterns:

First, use it as the default model. Most ordinary requests go to Flash first, giving both speed and cost control.

Second, design model routing. When Flash identifies a task as complex, high-risk, or requiring deep reasoning, escalate to Pro.

Third, use it for context compression. Before an agent executes a task, Flash can summarize documents, extract key facts, and generate structured context.

Fourth, make multimodal input part of the normal workflow. Images, screenshots, PDFs, audio, and video should not only be edge features; they can become default input types.

Fifth, evaluate with your own data. Do not rely only on official benchmarks. Test with your support questions, documents, code, images, and business workflows to decide which tasks Flash handles well and which need escalation.

Summary

The core positioning of Gemini 3.5 Flash is a multimodal workhorse model for high-frequency real applications. Its advantage is not replacing Pro-class flagship models, but placing speed, cost, long context, and multimodal ability into a tier better suited for large-scale invocation.

For developers, the most important part of Flash is not a single benchmark, but a product architecture shift: the default model can be faster, cheaper, and better at reading complex inputs; harder tasks can still escalate to stronger models. This keeps user experience good while controlling cost.

If Pro is the heavy tool for difficult problems, Flash is the main tool running on the production line every day. In real AI products, the latter is often what users experience most.

References:

Google official blog: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5/
Google DeepMind Gemini Flash: https://deepmind.google/en/models/gemini/flash/
User-provided Zhihu discussion link: https://www.zhihu.com/question/2040529179641385344/answer/2040531897613285214

After Google I/O, Should You Subscribe to GPT or Gemini? A Comparison for Regular Users and Developers

Thu, 21 May 2026 08:33:14 +0800

After Google I/O 2026, choosing an AI subscription has become more complicated.

The old question was simpler: for writing, Q&A, coding, and file analysis, most people looked at ChatGPT first; if they were deeply tied to Google Search, Android, Gmail, Docs, or YouTube, they would then consider Gemini. That has changed. At I/O, Google put Gemini 3.5 Flash, Gemini Omni, Antigravity 2.0, Gemini API Managed Agents, Google AI Studio, and AI Ultra into one broader subscription story. Gemini is no longer just an optional alternative; it has become a serious competing ecosystem.

This article does not compare abstract benchmark scores. It answers a practical question: should regular users, developers, content creators, and enterprise users subscribe to GPT / ChatGPT, or to Gemini / Google AI?

Note: AI subscription prices, quotas, regions, and model availability change quickly. This article was written on May 21, 2026. Before subscribing, always check the current OpenAI and Google pages.

The Short Answer

If you only want one primary subscription, use this logic:

Daily writing, Q&A, file analysis, office work, and mixed Chinese-English tasks: prioritize ChatGPT Plus.
Heavy coding, Codex usage, complex reasoning, and project-level code tasks: prioritize ChatGPT Plus / Pro, then decide whether to upgrade based on quota.
Deep use of the Google ecosystem, including Gmail, Docs, Drive, Android, and Search: prioritize Gemini / Google AI Pro.
Video, AI imagery, Google Flow, YouTube Shorts, and Gemini Omni: prioritize Google AI Pro / Ultra.
Antigravity, Gemini API Managed Agents, and workflows from AI Studio to Android: focus on Google AI Pro / Ultra.
Enterprise teams: do not compare only personal plans; look at Business / Enterprise, Workspace, permissions, audit, and data boundaries.
Limited budget: one paid primary subscription plus another platform’s free tier or pay-as-you-go API is usually better than two high-end subscriptions.

In one sentence: GPT is still the stronger default productivity and coding assistant; after Google I/O, Gemini looks more like a system-level AI suite inside the Google ecosystem.

What Changed for Gemini After Google I/O

Google I/O 2026 made Gemini’s value depend on much more than the Gemini App itself.

Several changes matter:

Gemini 3.5 Flash: Google positions it as a fast model for prompt-to-action workflows and real agent tasks.
Gemini Omni: creates content from arbitrary input, currently starting with video, with multimodal creation and natural-language iterative editing.
Google Antigravity 2.0: an agent-first development platform for multi-agent orchestration and coding.
Gemini API Managed Agents: lets developers create hosted agents that can reason, use tools, and execute code through the API.
Google AI Studio: moves from a prompt playground toward mobile, Android native app generation, and Antigravity project export.
Google AI Ultra: a new $100/month tier after I/O, aimed at developers, technical leads, knowledge workers, and advanced creators.

More importantly, Google moved Gemini App usage from traditional daily prompt limits toward a compute-used model. Complex video, code, and long-context tasks consume more quota, while simple text tasks consume less. Quotas refresh every five hours until weekly limits are reached.

That shows Google is trying to package Gemini subscriptions as an entry point for “model + app + creation + development tools + Google ecosystem.”

Who Is ChatGPT / GPT Best For Now?

ChatGPT remains very strong, especially for people who treat AI as a daily workhorse.

According to OpenAI’s current pricing page and help documentation, ChatGPT Free includes basic capabilities such as GPT-5.5 Instant. Plus provides GPT-5.5 Thinking, higher message and upload limits, stronger image generation, deep research, agent mode, projects, tasks, custom GPTs, and expanded Codex usage. Pro provides higher limits, GPT-5.5 Pro, higher Codex usage, and the largest deep research and agent mode capacity.

ChatGPT is especially suitable for:

Writing, summarizing, translation, and editing.
Complex Q&A and structured analysis.
File upload, spreadsheet analysis, and research reports.
Coding Q&A, code review, and refactoring advice.
Using Codex for repository-level tasks.
Multilingual content production.
Users who care about model quality and response stability but are not deeply tied to Google products.

For regular users, ChatGPT Plus is still the safest primary subscription. It covers a wide range of work, has a low learning curve, and handles Chinese and English tasks evenly.

For developers, the key part of ChatGPT is not only chat, but Codex. OpenAI’s help documentation says Codex can be used with eligible ChatGPT plans, with usage limits varying by plan. If you use Codex heavily for code edits, PRs, refactoring, or test fixes, you need to include Codex quota in your subscription decision.

Who Is Gemini / Google AI Best For Now?

After Google I/O, Gemini’s advantage is clearer: it is more deeply bound to the Google ecosystem.

Google AI subscriptions are no longer only model quota inside the Gemini App. They also include Gemini Omni, Google Flow, Antigravity, AI Studio, some YouTube Premium / Lite benefits, and Workspace / Android / Search ecosystem capabilities. Google also expanded AI Ultra into a $100 and higher-tier subscription line, emphasizing developers, technical leads, knowledge workers, and advanced creators.

Gemini is especially suitable if:

You deeply use Gmail, Docs, Drive, Sheets, Slides, and Android.
You want AI inside Google Search, YouTube, and Workspace.
You care about Gemini Omni, Google Flow, video generation, and video editing.
You want to try Antigravity, Gemini API Managed Agents, and AI Studio mobile.
You need ultra-long-context document understanding.
You build Google ecosystem apps, Android native apps, or Workspace automation.

Google’s help page says Gemini Apps context windows increase with subscription level: 32K without an AI plan, 128K with AI Plus, and 1 million with AI Pro and AI Ultra. AI Pro / Ultra also provides higher usage limits, more features, and some early access capabilities.

If your work already lives in the Google ecosystem, Gemini’s value becomes much larger. Otherwise, subscribing to Gemini only as “another chatbot” may not be more cost-effective than ChatGPT.

How Regular Users Should Choose

The easiest trap for regular users is subscribing to multiple platforms just because a new model was announced.

A more rational choice starts with your main use case.

If you mainly do:

Writing.
Research.
Summaries.
Reading PDFs.
Email.
Resume editing.
Language learning.
Daily Q&A.

Choose ChatGPT Plus first. It is more general-purpose, has clearer task boundaries, and does not require deep ecosystem lock-in.

If you mainly do:

Heavy Gmail / Docs / Drive / YouTube / Android use.
Want AI directly inside Google’s ecosystem.
Want to try Gemini App, Daily Brief, Google Search AI, and YouTube content Q&A.
Need long-context reading of Google documents.

Choose Google AI Pro first.

If you are a light user, start with the free tiers on both platforms and pay only after you clearly hit limits. Do not subscribe to a high-end plan just because you might use it someday.

How Developers Should Choose

Developers fall into two broad groups.

The first group mainly asks coding questions, fixes bugs, writes scripts, and reads repositories. For them, start with ChatGPT Plus / Pro + Codex.

Reasons:

Codex is tied to the ChatGPT account.
ChatGPT is stable for code explanation, refactoring, tests, and error analysis.
Plus already covers many daily development tasks.
Pro is better for high-frequency, long-running, complex repository tasks.

The second group builds around the Google ecosystem, agent platforms, Android, Workspace, or Gemini API. For them, start with Google AI Pro / Ultra.

Reasons:

Gemini 3.5 Flash is a key post-I/O model for agent workflows.
Antigravity 2.0 is Google’s agent-first development platform.
Managed Agents can create tool-using agents with isolated Linux environments through the API.
AI Studio connects more naturally with Android, Workspace, and Antigravity.

For full-stack developers, the most practical combination is usually:

ChatGPT Plus as the main tool for daily code and documentation.
Gemini free tier or AI Pro for Google ecosystem tasks, long context, and new video / agent capabilities.
Use APIs pay-as-you-go, and do not treat a personal subscription as a production API budget.

How Content Creators Should Choose

For content creators, the answer depends on what you create.

If you mainly do:

Copywriting.
Headlines.
Scripts.
Articles.
Image-and-text content.
Research organization.
Multilingual rewriting.

ChatGPT Plus is still very reliable.

If you mainly do:

Video generation.
Short-video ideas.
AI imagery.
YouTube Shorts.
Google Flow workflows.
Multimodal asset assembly.

Gemini / Google AI Pro or Ultra deserves more attention. After I/O, Gemini Omni and Google Flow are Google’s core offerings for creation.

If your budget is limited, subscribe to one text-first primary tool, then use the other platform’s free tier or a short-term subscription to test video capabilities. Video model quotas, queues, duration, resolution, and regional limits change quickly, so do not plan long-term production around them too early.

How Enterprises and Teams Should Choose

Enterprises should not choose like individual users.

What enterprises really need to examine is not “which model is stronger this week,” but:

Whether data is used for training.
Whether SSO, MFA, and RBAC are available.
Whether audit logs exist.
Whether internal knowledge connections are supported.
Whether plugins, connectors, and agent permissions can be controlled.
Whether the product meets compliance requirements.
Whether it integrates with the existing office suite.

If a company already heavily uses Google Workspace, Gemini enterprise plans are naturally worth evaluating. If the team has already built processes around ChatGPT, Codex, OpenAI API, and internal toolchains, OpenAI Business / Enterprise is the more natural fit.

Engineering teams also need to separately evaluate Codex, Antigravity, Gemini API Managed Agents, MCP, CI/CD, code permissions, repository access, and audit.

When You Need Pro / Ultra

Many people do not actually need a high-end tier.

Typical signs that you need ChatGPT Pro:

You use ChatGPT for long periods every day.
Plus limits are often insufficient.
You use Codex heavily.
You often run deep research, agent mode, and complex reasoning.
You need higher-end models such as GPT-5.5 Pro.

Typical signs that you need Google AI Ultra:

You use Gemini, Flow, and Antigravity frequently.
You need higher Gemini / Antigravity usage limits.
You create videos, AI imagery, or long-context research.
You deeply depend on the Google ecosystem and early access to new features.
You need Gemini Spark, Project Genie, or higher-tier subscription benefits.

If you only ask a few questions a day or occasionally write articles or edit code, Plus / Pro or AI Pro / Ultra may not be necessary.

The Most Cost-Effective Subscription Strategy

This combination is usually better:

Choose one paid primary subscription first.
Use the other platform’s free tier.
Pay for API only when you actually need API usage.
Turn high-consumption features such as video, agents, and deep research on and off monthly instead of subscribing all year blindly.
Review once a month: did you really use the quota?

Common combinations:

General office work: ChatGPT Plus + Gemini free tier.
Google ecosystem users: Google AI Pro + ChatGPT free tier.
Developers: ChatGPT Plus/Pro + Gemini API/AI Studio as needed.
Video creators: Google AI Pro/Ultra + ChatGPT free tier or Plus.
Enterprise teams: do not piece together personal plans; evaluate Business / Enterprise / Workspace plans directly.

Checklist Before Subscribing

Before paying, confirm these points:

Is the plan available in your region?
Is the model you need included in the plan?
Are Codex, Antigravity, Flow, and Omni actually available?
Do video features have region, age, queue, or resolution limits?
Is API usage included in the subscription, or billed separately?
Do file upload, context window, agent mode, and deep research have limits?
Do the privacy settings meet your project requirements?
Do you already have Google One, Workspace, ChatGPT Business, or school / company benefits?

Be especially careful: a personal subscription does not mean free API usage, unlimited commercial use, or enterprise compliance.

Summary

After Google I/O, Gemini is much more competitive, especially in video, multimodality, the Google ecosystem, Android, AI Studio, and Antigravity. But ChatGPT remains the steadier general-purpose choice, especially for daily writing, complex Q&A, file analysis, coding assistance, and Codex workflows.

The simplest judgment is:

If you do not know which to choose: start with ChatGPT Plus.
If you are a deep Google user: choose Google AI Pro.
If you are a heavy developer: compare Codex and Antigravity against your actual workflow.
If you are a video creator: look first at Gemini Omni, Flow, and Google AI Pro / Ultra.
If you are an enterprise user: choose by compliance, permissions, audit, and existing office ecosystem, not model hype.

More AI subscriptions are not automatically better. The more economical path is to define one primary workflow, then use other platforms as supplements instead of opening a long-term subscription after every product keynote.

References:

Google I/O 2026 Summary: Gemini 3.5, Omni, Antigravity, and System-Level Agents

Thu, 21 May 2026 00:07:06 +0800

The main line of Google I/O 2026 is clear: Google is moving Gemini from “model” and “chat assistant” into a fuller Agent ecosystem. It is not only answering questions. It is entering Search, Android, developer tools, video creation, shopping, Workspace, hardware, and enterprise platforms to help users complete longer task chains.

This article summarizes the main Google I/O 2026 announcements from official releases and a developer perspective. For real development, always follow the official Google, Android Developers, and Gemini API documentation.

One-Sentence Summary

The keyword for Google I/O 2026 is agentic Gemini era.

Google announced or strengthened several lines:

Gemini 3.5 Flash: speed, action capability, and Agent workflows.
Gemini Omni: creating content from any input, starting with video creation and editing.
Gemini app: moving from chat assistant to proactive, always-on, task-capable personal Agent.
Google Antigravity 2.0: evolving from an AI coding tool into an Agent-first development platform.
Gemini API Managed Agents: creating hosted Agents through APIs that can reason, use tools, and execute code.
Google AI Studio: expanding to mobile, native Android support, and project export to Antigravity.
Search, Shopping, YouTube, Workspace, and Android: all gaining stronger Gemini and Agent capabilities.

In other words, Google is no longer only showing “how smart the model is.” It is showing how models enter products, tools, and systems to actually execute tasks for users.

Gemini 3.5 Flash: From Prompt to Action

Gemini 3.5 is Google’s new model family at I/O 2026, with Gemini 3.5 Flash as the first public focus.

Google does not position it as simply a “faster chat model,” but as a high-speed engine for real Agent workflows. Google’s developer article describes 3.5 Flash as combining frontier intelligence and high speed to support the shift from prompt to action.

Its main significance:

Optimized for Agent and coding scenarios.
Supports longer task chains and tool use.
Available through Antigravity, Gemini API, Google AI Studio, Android Studio, Gemini Enterprise, and other entry points.
Better suited for applications that need fast responses, multi-turn execution, and frequent tool calls.

For developers, Gemini 3.5 Flash is not just another model option. It is one of the default engines for Google’s new Agent toolchain.

Gemini Omni: Video and World-Model Capabilities

Gemini Omni is another core I/O 2026 announcement. Google describes it as creating content from any input, with the current focus starting from video.

Its highlights fall into three areas:

Multimodal input: text, images, video, audio, and more can be used as references.
Video editing: users can modify video over multiple turns with natural language instead of stopping after one generation.
World understanding: it emphasizes consistency in physics, scenes, actions, narrative, and audiovisual output.

This means AI video tools are moving from “enter one prompt to generate a clip” toward “revise step by step as if talking to an editor.” For creators, the real value is not one-shot generation, but a controllable, traceable, and iterative editing process.

Gemini App: From Chat Assistant to Always-On Personal Agent

Google is also pushing Gemini app in a more Agent-like direction. Official posts describe Gemini app as becoming more proactive, offering daily briefs and always-on assistance.

Key points include:

Gemini 3.5 Flash entering Gemini app.
A new UI and more dynamic interaction.
Personal AI Agent concepts such as Gemini Spark.
Proactive daily briefs that organize what users need to know each day.
More emphasis on 24/7 background assistance instead of waiting for the user to start every chat.

This is the part that affects ordinary users most. Gemini used to feel more like a “you ask, I answer” assistant. After I/O 2026, Google wants it to feel more like a personal Agent that follows up on tasks, proactively reminds users, and works across products.

Antigravity 2.0: Developer Tools Become Agent-First

One of the most important developer-side announcements is Google Antigravity 2.0.

Google positions Antigravity as an agent-first development platform. After I/O 2026, it is not only helping developers write code. It is meant to help developers move from ideas and prototypes to Agent orchestration and production delivery.

Core changes listed by Google include:

Antigravity 2.0 standalone desktop app.
Multi-Agent parallel orchestration.
Dynamic subagents.
Background scheduled tasks.
Integration with Google AI Studio, Android, Firebase, and related ecosystems.
Antigravity CLI for terminal users.
Antigravity SDK for custom Agent behavior and deployment.

This shows that AI coding tools are entering the next stage after “code completion / conversational generation”: developers will manage multiple executable Agents, not just one chat window.

Gemini API Managed Agents: Hosting Agents as API Capabilities

Google also introduced Managed Agents in the Gemini API.

According to the official description, these Agents can be created with a single API call. They can reason, use tools, and execute code in an isolated Linux environment, supported by the Antigravity agent harness.

This matters to developers:

You do not need to build the full Agent runtime yourself.
You can get a persistent, isolated execution environment.
Multi-turn interactions can preserve files and state.
Agents can be extended with markdown skills, custom instructions, and templates.
They are available through Interactions API and Google AI Studio.

If this line matures, Agent platforms will increasingly look like cloud services: developers will not only call models, but call Agents with state, tools, execution environments, and security boundaries.

Google AI Studio: From Prompt Playground to App Generation Entry Point

At I/O 2026, Google AI Studio also moves further.

Key changes include:

Google AI Studio mobile app for capturing ideas and generating prototypes on mobile.
Workspace API integration, making it easier for Agents to access Google Workspace.
Project export to Antigravity, carrying context into local development and production work.
Native Android support, allowing users to build Android apps from prompts.
Google Play Console integration to publish apps to test tracks.

This turns AI Studio from “a place to tune prompts and test models” into an entry point from idea to app. Its relationship with Antigravity is clearer too: AI Studio is good for fast ideation and generation, while Antigravity is better for continued development, orchestration, debugging, and delivery.

Android and AppFunctions: Key Interfaces for Mobile Agents

Android system-level Agents are worth watching on their own, but they need to be understood through accurate interfaces and product boundaries.

The most important current piece is Android’s official AppFunctions. The official documentation describes AppFunctions as an Android platform API with Jetpack libraries that lets apps expose their capabilities to agents, assistants, and other authorized callers. It also simplifies Android MCP integration.

Its significance is that mobile automation no longer has to rely only on screenshots, OCR, simulated taps, and UI control positioning.

Traditional mobile automation looks like:

Recognize the screen.
Find the button.
Simulate a tap.
Wait for the page to change.
Retry after errors.

The AppFunctions direction is:

Apps declare what they can do.
Agents call those capabilities with authorization.
The system handles permissions, call boundaries, and security constraints.

This will affect Android app design. Future apps will not only need human-facing UIs, but also core capabilities designed as Agent-callable interfaces.

Search, Shopping, and Content Products Are Becoming Agentic Too

Google I/O 2026 changes are not limited to models and developer tools. Search and consumer products are changing at the same time.

Official I/O summaries mention:

Search entering a new AI Search stage.
Information agents appearing in Search.
Gemini Spark and Daily Brief entering Gemini app.
Universal Cart making shopping carts smarter.
Ask YouTube enabling conversational queries and navigation over video content.
Gemini capabilities expanding to more products and form factors.

These announcements show that Google’s Agent direction is not a single product. It is spreading horizontally across search, video, shopping, productivity, mobile, and hardware scenarios.

Practical Impact for Developers

The biggest impact of Google I/O 2026 for developers is not “another model.” It is that the development target is changing.

Developers used to mainly build:

Apps.
Websites.
APIs.
Plugins.
Automation scripts.

Next, they will also build:

App capabilities callable by Agents.
Multi-Agent workflows.
Stateful tool execution environments.
Auditable automation flows.
Human-in-the-loop confirmation mechanisms.
Integrations with MCP, AppFunctions, Workspace API, Playwright, Firebase, and other tools.

Software will increasingly look like a set of capabilities, not only a set of interfaces. Products that expose their capabilities clearly, reliably, and safely to Agents will be more likely to enter users’ automation task chains.

Impact on Mobile Automation

Mobile automation will gradually move from “GUI first” to “API first, GUI as fallback.”

In the short term, screenshot recognition, OCR, simulated taps, and browser automation still matter because many older apps have no standard interface.

In the long term, if Android AppFunctions, MCP, and system-level permission models mature, stable task execution will lean toward:

First calling capabilities declared by apps.
Then calling system interfaces when needed.
Then using GUI automation as a fallback.

This will change RPA, mobile Agents, testing tools, and app ecosystems. Apps that expose capabilities are easier for system-level Agents to call. Apps that do not may still only be operated by the old “look at screen, tap screen” approach.

Security, Permissions, and Auditing Become Hard Requirements

The stronger Agents become, the higher the risk.

If an Agent can execute tasks across apps, make payments, change settings, access files, and read context, it needs clear security boundaries:

Permission levels.
Explicit user authorization.
Secondary confirmation for sensitive actions.
Sandbox isolation.
Operation logs.
Reversibility and rollback.
Enterprise auditing and compliance.

This is why Google emphasizes isolated environments for hosted Agents, permission requirements for AppFunctions, enterprise platforms, and controlled deployment. The future of Agents is not “do anything without limits,” but executable, traceable, and governable behavior inside security boundaries.

Summary

The main content of Google I/O 2026 can be summarized in one sentence: Google is turning Gemini into an Agent platform spanning models, apps, systems, developer tools, and hardware.

Gemini 3.5 Flash provides speed and action capability. Gemini Omni pushes multimodal creation toward video and world understanding. Gemini app becomes a proactive personal assistant. Antigravity 2.0 and Managed Agents push developer tools toward Agent-native development. AppFunctions lets Android apps begin exposing capabilities to intelligent agents.

For developers, the next thing to watch is not only model parameters, but how to structure application capabilities, connect to Agent toolchains, design permissions and auditing, and make products safely and reliably callable in a system-level Agent ecosystem.

References:

How to Use Gemini 3.5 Flash and Gemini Omni for Free: Entry Points for Users and Developers

Wed, 20 May 2026 23:13:35 +0800

After Google released Gemini 3.5 Flash and Gemini Omni, the practical question is not the benchmark score, but how ordinary users and developers can actually use them, which entry points are free, and which ones are only low-friction trials.

The short version:

For chat, writing, image understanding, and everyday Q&A: use Gemini app first.
To test Gemini 3.5 Flash parameters, prompts, and multimodal input: use Google AI Studio.
To call Gemini 3.5 Flash from code: create an API key in AI Studio.
To try it from the terminal for free: look at Gemini CLI.
To try Gemini Omni video editing: start with Gemini app and Google Flow.
For real production use: do not rely on free quotas; move to a paid API or Vertex AI.

Note: free quotas, regional availability, subscription tiers, and model menus change over time. This article was written on May 20, 2026. Before official use, always check Google’s current pages.

Free Gemini 3.5 Flash Method 1: Gemini App

The simplest entry point is Gemini app:

https://gemini.google.com/

The basic flow is straightforward:

Open Gemini.
Sign in with a Google account.
Look for 3.5 Flash in the model selector.
Start chatting.

This entry point is best for ordinary users. You can use it for writing, summarization, image understanding, file analysis, everyday Q&A, and simple planning. According to public reports, Gemini 3.5 Flash has been made available to users globally and can be selected from Gemini’s model dropdown.

The limits are also clear: free users usually face daily message, regional, and feature limits. If you exceed the limit, you need to wait for the quota to refresh or upgrade your subscription.

Free Gemini 3.5 Flash Method 2: Google AI Studio

If you want more than chat, and need to tune prompts, inspect parameters, or test structured output, Google AI Studio is a better fit:

https://aistudio.google.com/

Basic flow:

Sign in to Google AI Studio.
Create a new prompt.
Select gemini-3.5-flash in the model dropdown.
Enter the prompt and run it.

AI Studio gives you more control. You can adjust temperature, system instructions, structured output, and multi-image input, and you can export a working prompt into code or an API call.

For developers, AI Studio is a free testing bench. Tune the prompt and input format here first, then move into API integration to avoid wasting quota.

Free Gemini 3.5 Flash Method 3: Free API Key

Developers care most about the API. AI Studio can create a Gemini API key for calling gemini-3.5-flash.

Basic flow:

Open Google AI Studio.
Find Get API key.
Select or create a project.
Create an API key.
Save the key to a local environment variable.

Python example:

import os
from google import genai

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

response = client.models.generate_content(
    model="gemini-3.5-flash",
    contents="Explain in three sentences what Gemini 3.5 Flash is best suited for."
)

print(response.text)

Node.js example:

import { GoogleGenAI } from "@google/genai";

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const response = await ai.models.generateContent({
  model: "gemini-3.5-flash",
  contents: "Explain in three sentences what Gemini 3.5 Flash is best suited for."
});

console.log(response.text);

curl example:

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash:generateContent" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"contents":[{"parts":[{"text":"Hello Gemini 3.5 Flash"}]}]}'

Public information suggests that the AI Studio free tier usually gives Gemini Flash models a daily request allowance. The exact numbers can vary by time, region, and account status. Common claims include around 1,500 requests per day, per-minute request limits, and token limits. Do not bake those numbers into a production plan; check Google’s current pricing and limits pages before launch.

Free Gemini 3.5 Flash Method 4: Gemini CLI

If you like the command line, look at Gemini CLI. It is useful for temporary scripts, repository summaries, file reading, and quick Q&A in the terminal.

Installation is usually:

`1`	`npm install -g @google/gemini-cli`

Then run:

gemini

The CLI is better suited to personal developer workflows than production integration. Production should still use API keys, service accounts, permission controls, and auditable calling patterns.

Free or Low-Friction Gemini Omni Access: Gemini App and Google Flow

Gemini Omni is a multimodal model for video creation and editing. Its core capability is not ordinary text Q&A, but multi-turn video editing with natural language while referencing image, text, video, and audio inputs.

The Google DeepMind page lists these entry points:

Gemini app.
Google Flow.
YouTube Shorts.

The page also notes that a Google AI subscription is required, and that features vary by subscription tier and region. So “free access” to Gemini Omni should be understood more carefully: some entry points may let free users see or try part of the experience, but full video editing may require a subscription, regional availability, or product rollout access.

If you only want to try it, use this order:

Open Gemini app first and check whether Gemini Omni or a related video editing entry is available.
Then open Google Flow: https://flow.google/
If you make short-form content, watch for Omni-related editing features in YouTube Shorts.

If the entry point is not visible, it usually does not mean you did something wrong. Your account, region, subscription tier, or rollout group may simply not qualify yet.

How Gemini Omni Is Best Used

Gemini Omni is more suitable for creators than for ordinary chat.

You can try these directions:

Upload or select a video and ask it to change the style.
Make a specific action in the video more exaggerated.
Use a reference image to replace an object or character in the scene.
Modify camera, action, environment, and style over multiple turns.
Combine sketches, reference images, audio, or video into a new output.

You can write prompts as if giving instructions to an editor:

Keep the original person and room structure unchanged. Change the effect after touching the mirror into liquid ripples. The motion should feel natural, and the lighting should not change abruptly.

For multi-turn editing, do not pack too many requirements into one request. A safer approach is:

Change the main action first.
Then change the style.
Then adjust the camera angle.
Finally tune sound, text, and rhythm.

This makes consistency easier to maintain and helps you identify which step caused a problem.

Common Pitfalls When Using Free Access

First, free quota is not production quota. A free API key is suitable for testing, personal tools, and prototypes, not for promising a stable service.

Second, do not send sensitive data to free or third-party entry points. This includes private code, customer data, contracts, keys, financial spreadsheets, and internal documents.

Third, check data-use settings. Free tiers may have different data-use policies, so review the settings in AI Studio or your Google account before use.

Fourth, video capabilities are usually more restricted than text capabilities. Gemini Omni-style video editing may be limited by subscription, region, queueing, duration, resolution, and content safety policies.

Fifth, be careful with third-party “unlimited free API” services. Many gateways rate-limit, forward requests, keep logs, or use opaque payment methods. Sensitive work should not go through these entry points.

Which Entry Point Should You Choose?

If you are an ordinary user:

Gemini 3.5 Flash: use Gemini app.
Gemini Omni: check Gemini app first, then Google Flow.

If you are a creator:

Use Google Flow to try Omni video workflows.
Use Gemini app for scripts, storyboards, prompts, and material descriptions.

If you are a developer:

Use AI Studio to debug prompts.
Use an API key to integrate gemini-3.5-flash.
Use Gemini CLI for personal terminal workflows.
For production, consider Vertex AI or the paid API.

If you are an enterprise:

Do not rely on free quotas.
Focus on permissions, logs, audits, data residency, compliance, and key management.
For video generation and editing, add watermarking, content review, and copyright processes.

Summary

Gemini 3.5 Flash has relatively clear free access paths: Gemini app, Google AI Studio, AI Studio API key, and Gemini CLI can all serve as low-friction entry points. It is suitable for chat, writing, coding, agent prototypes, and multimodal testing.

Gemini Omni focuses on video editing and multimodal creation. Its main entry points are Gemini app, Google Flow, and YouTube Shorts, but full capabilities are more likely to depend on subscription and region. It is best for creators to start with trials and concept validation, not to plan around it as a stable production service from day one.

The safest strategy is: test text and code tasks first with the Gemini 3.5 Flash free tier; validate video creation effects with Gemini Omni in Gemini app or Flow; when you need to launch something real, move to a formal setup with auditability, billing, and controlled permissions.

References:

Gemini 3.5 Is Here: Flash Leads as Google Focuses on Agents and Long-Running Tasks

Wed, 20 May 2026 22:51:31 +0800

Google officially released the Gemini 3.5 series on May 20, 2026. The first model available is Gemini 3.5 Flash. Its positioning is not just chat, but agents, code generation, and long-running complex task execution.

The message is clear: Google wants Gemini 3.5 to answer questions, but also to plan, execute, check results, and keep work moving across multi-step workflows.

Gemini 3.5 Flash Comes First

Gemini 3.5 Flash is already available to several groups:

General users can try it in the Gemini app and AI Mode in Google Search.
Developers can use it through Google Antigravity, Google AI Studio, and the Gemini API in Android Studio.
Enterprise users can access it through Gemini Enterprise Agent Platform and Gemini Enterprise.

Google also said Gemini 3.5 Pro is still in development, already being used internally at Google, and expected to launch next month.

This means the 3.5 series will continue the Flash and Pro split: Flash emphasizes speed, cost, and scalable execution, while Pro will likely target more complex and higher-capability use cases.

The Focus Is Agents and Coding

Google describes Gemini 3.5 Flash as one of its strongest models for agents and coding. The announcement says it beats some Gemini 3.1 Pro results on coding and agent benchmarks such as Terminal-Bench 2.1, GDPval-AA, MCP Atlas, and CharXiv Reasoning.

Most users do not need to care about every benchmark number. The more important point is that Google is pushing model capability toward executable workflows: not only writing code, but also migrating old projects, developing complex apps, organizing financial reports, analyzing data, and running repeated tests.

In the Antigravity development framework, Gemini 3.5 Flash can use multiple collaborating subagents to handle large tasks. Google showed examples such as reading the AlphaZero paper and building a playable game, converting legacy code to Next.js, and generating cityscapes and UI options in parallel.

The direction is clear: AI coding tools are moving from “generate a piece of code” toward “coordinate multiple agents to complete a project.”

Stronger Multimodal UI and Graphics

Gemini 3.5 Flash builds on Gemini 3’s multimodal foundation. Google says it can generate richer web UIs, interactive animations, and visual content.

The announcement includes examples such as:

Creating interactive animations for research papers.
Turning text descriptions into interactive hardware models.
Generating a complete brand concept for a school fundraiser.
Producing multiple UX options for a checkout flow in a short time.

This matters for developers and product teams. The model is no longer only writing explanations. It can participate in frontend prototypes, interaction design, and visualization work.

Enterprise Use: Automating Time-Consuming Workflows

Google listed several partner examples. Shopify uses subagents to analyze complex data and forecast merchant growth. Macquarie Bank is testing 3.5 Flash on documents over 100 pages to accelerate account-opening workflows. Salesforce is integrating it into Agentforce. Ramp uses it to improve OCR for complex invoices. Xero uses AI agents for administrative workflows. Databricks uses automated workflows to monitor data anomalies and suggest fixes.

These examples point to the same trend: enterprise adoption of large models is moving from one-off Q&A to workflow automation. Whether a model is inexpensive, fast, and stable over long tasks can matter more than whether one answer looks impressive.

Gemini Spark: A Personal AI Agent

Google also announced Gemini Spark, a personal AI agent powered by Gemini 3.5 Flash. Its goal is to run over long periods and proactively perform tasks under user guidance.

Gemini Spark has started rolling out to trusted testers. Google plans to open a beta next week to Google AI Ultra subscribers in the United States.

This is worth watching. Google Search, the Gemini app, Android, Workspace, and browser-related ecosystems already touch many parts of personal digital life. If a personal agent can connect with these entry points, its impact may be larger than a standalone chatbot.

Safety Moves Further Upstream

Google says Gemini 3.5 was developed under its Frontier Safety Framework, with strengthened protections for information security and CBRN-related risks. The announcement also mentions interpretability tools that help examine and understand model reasoning before responses are delivered.

This shows that frontier model releases are no longer only a capability race. The more a model emphasizes agents, autonomous execution, and long-running tasks, the more important safety controls, false refusal rates, harmful-output prevention, and interpretability become.

How to View Gemini 3.5

Gemini 3.5 Flash is not just another model launch. It looks more like Google’s bet on the next shape of AI products: models that can call tools, split tasks, coordinate execution, generate UIs, and enter personal and enterprise workflows.

For developers, the important things to watch are the real experience in Google Antigravity, AI Studio, the Gemini API, and Android Studio. For enterprises, the question is whether it can reliably reduce manual work in real workflows, not just score well on benchmarks.

Gemini 3.5 Pro is not publicly available yet. Once Pro ships, the differences between Flash and Pro in capability, price, speed, and context handling will decide which production scenarios each model fits best.

References:

Google Blog: Gemini 3.5

Google Gemini Spark Leak: A 24/7 Gemini Agent May Be Coming

Sun, 17 May 2026 11:58:08 +0800

Google has not officially released Gemini Spark.

Current information about it mainly comes from internal Gemini Web test screens, community screenshots, TestingCatalog reporting, and 36Kr / Xinzhiyuan’s summary of related leaks. The consistent picture is that Gemini Spark BETA may be an always-on AI Agent that Google is preparing. Its positioning is no longer just a chat assistant, but an “everyday AI agent” that can handle email, online tasks, and multi-step workflows in the background.

So the boundary should be clear first: this is a leak analysis, not an official Google announcement. All features, naming, and launch timing still need to be confirmed by Google.

Bottom line

Based on currently exposed information, Gemini Spark has three key points:

It may be a 24-hour online Agent inside the Gemini system, not a normal chat model.
It may use broader personal context, including Google apps, chat history, tasks, logged-in websites, and location.
Its risks are as large as its appeal, because it may involve information sharing, remote browser data, purchases, and third-party service calls.

If Google really launches Spark, Gemini’s role will change from “AI that answers questions” to “AI that continuously handles tasks for you.”

What Gemini Spark is

TestingCatalog reported on May 14, 2026 that Google is testing Gemini Spark BETA inside Gemini Web. The exposed welcome text describes it as an everyday AI agent that can help users 24/7 with inbox, online tasks, and more multi-step work.

The 36Kr / Xinzhiyuan article also says that after Spark was uncovered, the outside world saw a “full-time Agent” direction: it can stay on standby all day, process inboxes, execute online tasks, and may even involve purchases and information sharing.

This means Spark is not simply a new model name. It looks more like a Gemini product-layer upgrade: bringing Gemini out of the conversation window and into users’ email, web, calendar, tasks, and cross-app workflows.

How it may work

According to the hidden onboarding text disclosed by TestingCatalog, Gemini Spark may gather context from multiple sources, including:

Connected Apps.
skills.
chats.
tasks.
Websites the user has logged into.
Personal intelligence.
location.

This information would help Spark understand what the user wants to complete and call the necessary context while executing tasks. The text also says that, to complete some actions, Gemini may share necessary information with third parties, such as names, contact details, files, preferences, and information the user may consider sensitive.

If these descriptions prove accurate, Spark will work more like a context-aware agent system than a one-shot Q&A assistant. It will not only look at the current prompt, but may combine long-term preferences, connected apps, browser state, and task history.

Why it matters

The key to Gemini Spark is not one more chat entry point. It is that Google has a natural ecosystem entry point.

OpenAI and Anthropic can build powerful Agents, but they do not naturally own the full chain of Gmail, Calendar, Drive, Chrome, Android, and Workspace. If Google connects Spark into these products, users will not need to build many extra workflows before letting an Agent enter daily work.

This may bring three changes.

First, Gemini may move from passive Q&A to active execution. Users may no longer only ask “summarize this email”; they may ask it to continuously organize the inbox, track tasks, and take follow-up actions.

Second, Agents will rely more on personal context. The more it understands your email, calendar, files, browser state, and preferences, the more useful the result may be.

Third, permission boundaries will become more sensitive. Doing more also means users need to know more clearly when it can act, how far it can go, and whether confirmation is required.

Where the risks are

Several details in the onboarding text disclosed by TestingCatalog are worth watching.

First, Spark is experimental. Even if it launches, it should not be treated as a fully mature system that needs no supervision.

Second, although the system is designed to ask for permission before sensitive operations, the text also warns that it may share information or complete purchases without asking.

Third, to maintain session continuity, Gemini will save remote browser data, such as login details and remote code execution data. Users can clear these data in Settings and can also disable Connected Apps and Personal intelligence.

Taken together, these points show that Spark’s product direction is aggressive: it wants to be an Agent that can truly execute tasks, not only generate suggestions. But the closer it gets to real execution, the more it needs strict permissioning, auditing, confirmation, and rollback mechanisms.

Relationship with Remy and AI Ultra

TestingCatalog says Spark may be a renamed version of the agentic Gemini upgrade previously codenamed Remy, and may also relate to the Gemini Agent direction for Google AI Ultra subscribers.

If this clue is correct, Spark may not be a brand-new project from nowhere. It may be Google repackaging previously higher-end or more closed Agent capabilities and preparing to bring them to a wider audience.

36Kr / Xinzhiyuan also describes it as an upgrade from “Remy” to “Spark”: Gemini Agent is no longer just a feature, but is moving toward a 24/7 digital life manager.

But this is still a judgment based on leaked information. Whether Google will use Spark as the official name, whether it will be limited to AI Ultra, and whether a lighter subscription tier will appear all need official confirmation.

MCP, skills, and the tool ecosystem

The same batch of community screenshots also showed model selector entries such as MCP Tool Testing. The 36Kr article suggests this may hint that the new Gemini will natively support MCP third-party tool integration, with Thinking mode also being rebuilt.

This clue becomes more interesting when viewed together with Spark.

If Spark were only a chat assistant, skills and MCP would matter less. But if Spark is a long-running Agent, it needs to reliably call tools, access web pages, execute tasks, read and write context, and deliver results to users.

In other words, Spark may not be a single feature. It may be part of Google’s Agent tool ecosystem: the model handles understanding and planning, while skills / MCP / connected apps handle execution and expansion.

What it means for ordinary users

If Gemini Spark really launches, ordinary users may see these direct changes:

Email is not only summarized, but can be categorized, followed up, and turned into tasks.
Web tasks are not only suggested, but may be continuously executed in a remote browser.
Calendar, location, preferences, and previous chats become long-term Agent context.
Purchases, bookings, form filling, and similar actions may enter the AI execution range.

This sounds convenient, but users will need new habits: not only checking what the AI says, but also what it is preparing to do, what it has already done, whether it can be undone, and whether there is a record.

Future AI Agent experience may depend not only on model intelligence, but also on clear permission prompts, inspectable task logs, and recovery from mistakes.

What it means for developers and teams

For developers, Spark matters because Google may be moving Agents from “demo products” toward real workflow platforms.

If Spark can reliably connect Google apps, third-party tools, and browser state, developers will care about:

Whether APIs or extension mechanisms are open.
Whether MCP or skills can be connected by third parties.
Whether enterprise admins can control permissions, data retention, and audits.
Whether Agent execution failures have traceable logs.
Whether sandboxing, approval flows, and sensitive-operation confirmation are supported.

For teams, Spark may first enter high-frequency scenarios such as Gmail, Calendar, Docs, Drive, and Chrome. It may not be suitable for fully automated high-risk work at the beginning, but it is a strong fit for inbox triage, meeting follow-up, document organization, market research, and lightweight operations tasks.

How to read it now

This story is best understood as “high-confidence direction, low-certainty details.”

The high-confidence direction is that Google is pushing Gemini Agents to be more proactive, longer-running, and more deeply integrated with its ecosystem. The Gemini Web test text reported by TestingCatalog, community screenshots, and 36Kr’s summary of multiple leaks all point in the same direction.

The low-certainty details are the official name, launch timing, permission rules, subscription tiers, supported regions, API availability, and whether it will really be called Gemini Spark.

The safest view for now:

Do not treat Spark as an already released official product.
Treat it as a strong signal for Google’s next AI Agent direction.
Wait for official explanations around permissions, privacy, third-party data sharing, and remote browser data storage.

Summary

If Gemini Spark eventually launches, it may be a key step in Gemini’s move from chat assistant to always-on Agent. It is not just a model swap; it places Gemini into Google’s ecosystem of email, web, tasks, location, personal intelligence, and third-party services.

Its potential is large: more proactive, closer to real workflows, and easier to distribute to many users through Google’s ecosystem. Its risks are just as large: once AI can share information, save browser state, make purchases, and call third-party services, permission boundaries must be extremely clear.

So the most important question about Gemini Spark is not “how smart is it”, but how Google plans to make a 24-hour online AI Agent controllable, auditable, and trustworthy.

References:

Gemini 3.5 Pro Leak: Codenamed Cappuccino, Google Tries to Regain Momentum in Coding and Agents

Sun, 17 May 2026 11:47:27 +0800

Google has not officially released Gemini 3.5 Pro.

What we can see so far mainly comes from developer community screenshots, anonymous benchmarks, leakers, and media reports. On May 15, 2026, 36Kr / Xinzhiyuan reported that a next-generation Gemini checkpoint may be internally codenamed Cappuccino, and that related models have already surfaced in communities and benchmark platforms.

This information should not be treated as an official launch, but it points in a clear direction: Google is trying to address two gaps at once, coding and reasoning on one side, and always-on AI agents on the other.

Bottom line

This leak can be read in three layers:

Gemini 3.5 Pro has not been officially released, and Cappuccino looks more like an internal checkpoint or candidate build.
The leaked information suggests the new Gemini is improving in code generation, SVG / interactive web generation, and multimodal output.
Google’s parallel test of Gemini Spark may matter more than the model itself, because it points to a 24-hour personal AI agent.

In other words, this is not just a “model benchmark” story. It looks more like a product roadmap signal ahead of Google I/O: the model needs to catch up with GPT-5.5, while the agent layer needs to capture user workflows.

What Cappuccino is

The 36Kr article says a post from Lentils indicates that the Gemini 3.5 Pro checkpoint codenamed Cappuccino has started to appear. The community had been discussing Gemini 3.2 only hours earlier, but the latest leak jumped directly to 3.5.

If that naming is ultimately accurate, Google may want to frame the next Gemini as a larger version jump rather than a routine point release.

For now, Cappuccino should still be treated as a leaked internal codename. It does not mean Google has publicly launched the final model, and it does not guarantee that the final release name will be Gemini 3.5 Pro.

Why coding is the focus

The most discussed part of the leak is the new Gemini’s coding ability.

According to community screenshots and benchmark claims cited by 36Kr, the new model appears stronger at:

Generating SVG and visual components.
Generating interactive web apps.
Handling animation, 3D, adjustable control panels, and other complex frontend outputs.
Improving logical reasoning and code generation.

The article also cites Abacus.AI CEO Bindu Reddy as saying that 3.2 Flash is close to GPT-5.5 in coding and reasoning while being much cheaper. Other media sources reportedly believe the new Gemini roughly reaches the GPT-5.5 tier overall, but may not represent a qualitative leap.

That is why the phrase “matches GPT-5.5” needs caution. It is more of a relative judgment from different leaks and anonymous tests than an official Google benchmark result.

Why Google needs to catch up in coding

AI coding has moved from developer tooling into the center of foundation model competition.

OpenAI has Codex, and Anthropic has Claude Code. They serve engineers, but they also bring product managers, designers, and operators into workflows where natural language can produce runnable products.

By comparison, Google has Gemini and Antigravity, but it has not formed the same default entry point in developer mindshare. The 36Kr article also notes that Antigravity has not truly broken through externally, and that pricing, quota reminders, and experience stability have drawn community discussion.

So if the new Gemini needs to prove itself, coding is the most direct battlefield. The question is not only whether it can write code, but whether it can reliably produce complete interfaces, understand complex requirements, call tools, fix errors, and fit into real development workflows.

Spark may matter more than 3.5 Pro

In the same wave of leaks, Gemini Spark BETA also surfaced.

According to TestingCatalog and other sources, Spark is positioned like an always-on AI agent: it can process inboxes, execute online tasks, manage multi-step workflows, and connect context from Google apps, skill modules, chat history, scheduled tasks, logged-in websites, and location data.

That means Spark is not a normal chat entry point. It may be a system that stays online, continuously reads context, and performs tasks for users.

Its appeal is obvious: if Google can connect Gmail, Calendar, Chrome, Android, Workspace, and Gemini, Spark will have a distribution advantage that OpenAI and Anthropic cannot easily copy.

The risk is just as obvious. The 36Kr article mentions wording around Spark saying it may share information or complete purchases without asking. Even if the system is designed to request permission before sensitive operations, this kind of agent still raises privacy, authorization-boundary, and accidental-action risks.

What this means for ordinary users

If you are a regular Gemini user, the most important part of this leak is not the model name. It is three shifts.

First, Google may continue to strengthen the ability to produce complete results. Users have often complained that Gemini can be lazy with visual generation, SVG, and frontend pages. If the new model can produce several complete options in one pass, the experience will improve noticeably.

Second, coding ability may continue to move into lighter models. The leak repeatedly mentions Flash improvements in coding, reasoning, and interactive generation, which means complex tasks may not always require Pro models in the future.

Third, agents will become more proactive. If Spark launches, Gemini may no longer just answer questions. It may start taking over email, web tasks, purchases, calendars, and cross-app workflows over longer periods.

That is good for efficiency, but it creates a new challenge for permission management.

What this means for developers

Developers should watch two issues more closely.

The first is tooling. The 36Kr article says community screenshots showed an unreleased entry called MCP Tool Testing in the model selector. If Gemini natively supports MCP or third-party tool testing, it will be easier to connect it to developers’ own toolchains.

The second is cost and stability. Even if the new Gemini matches GPT-5.5 on some benchmarks, developers will ultimately judge three things: actual code quality, context stability, and whether pricing and quotas are predictable.

The past year of AI coding tool competition has shown that model capability is only the ticket in. What keeps developers is whether the tool can reliably edit code, run tests, read context, and handle edge cases in daily projects.

How to read this news now

This story is best understood as “strong signal, weak confirmation.”

The strong signal is that multiple community clues point to Google preparing a stronger new Gemini and a more proactive Gemini Spark Agent.

The weak confirmation is that Gemini 3.5 Pro has not been officially released, Cappuccino remains a leaked codename, and claims that it “matches GPT-5.5” still need validation through official Google benchmarks, third-party tests, and real user experience.

The safest view for now:

Do not treat it as a released product.
Treat it as an early preview of Google’s next Gemini direction.
Watch whether I/O or later official events confirm the model name, API availability, pricing, context window, tool calling, and agent permission boundaries.

Summary

The exposure of Gemini 3.5 Pro / Cappuccino suggests Google may be preparing a stronger next-generation Gemini push. It is not trying to fix one isolated capability, but a whole AI workflow: the model needs to write code better, generate interfaces, and handle complex reasoning, while Spark pushes Gemini toward an always-on agent.

But before an official release, all benchmarks and screenshots remain clues. What will decide whether Gemini 3.5 Pro can regain momentum is not whether the codename sounds good, but whether it can reliably win in real development, real office work, and real multi-step tasks.

References:

Gemini Intelligence on Android: Google Is Turning the Phone into a Proactive AI System

Sun, 17 May 2026 09:13:32 +0800

On May 12, 2026, Google published “A smarter, more proactive Android with Gemini Intelligence,” introducing Gemini Intelligence on Android. This is not a standalone chat app. It brings Gemini capabilities into Android, Chrome, Gboard, Autofill, widgets, and multi-device experiences, moving the phone from “wait for the user to tap” toward “proactively help the user complete tasks.”

In short, Google wants Android to move from an operating system toward an intelligence system. The phone no longer just opens apps, shows notifications, and runs settings. It can understand the screen, apps, voice, and personal context, then complete more complex actions with user confirmation.

Short Version

Gemini Intelligence on Android focuses on five areas:

Multi-step automation: Gemini can complete flows across apps, such as rides, shopping, and research.
Smarter Chrome browsing: summarize pages, compare information, and handle some repetitive web tasks on Android.
Upgraded Autofill: use Gemini and personal context to fill more complex forms.
Rambler: turn natural speech into clearer, more polished text.
Natural-language widgets: describe what you want, and Android generates custom widgets.

These features will start rolling out in summer 2026, first on select Samsung Galaxy and Google Pixel phones, and later to more Android devices including watches, cars, glasses, and laptops.

Multi-Step Automation: From Suggestions to Execution

The most important direction is letting Gemini complete multi-step tasks across apps.

Google gives examples such as booking a spin class, finding a course syllabus in Gmail and adding required books to a shopping cart, or seeing a travel poster and asking Gemini to find a similar trip on Expedia.

The hard part is not just understanding one sentence. The system needs to understand:

What is on the user’s current screen or image.
App information the user has authorized.
Which app should be opened next.
Which steps can be automated.
Which steps must pause for user confirmation.

Google emphasizes that Gemini acts on user instructions and stops when the task is done, with final confirmation remaining under user control. This is not a fully autonomous agent, but a mobile agent with human confirmation in the loop.

Screen and Image Context Matter More

One important change is screen context and image context.

Older phone assistants mostly relied on voice commands and fixed app integrations. Gemini Intelligence puts more emphasis on “seeing” the current screen. For example, if a user has a shopping list in notes, they can long-press the power button to summon Gemini and ask it to create a delivery cart from the list.

This means Android AI is not just a chatbot. It is trying to understand the user’s current operating environment. Future mobile AI competition may depend not only on who has the better model answer, but also on:

Whether the AI can understand the current screen.
Whether it can act across apps.
Whether it can track task progress in the background.
Whether it can reliably ask for confirmation at key points.

That is a major difference between mobile AI and web chat AI.

Chrome: From Search to Web Task Agent

Google says Android devices will get a smarter Gemini in Chrome starting in late June 2026.

It can help users research, summarize, and compare web content, and Chrome auto browse can handle some repetitive web tasks such as appointments and parking reservations.

This means Gemini in Chrome is not just a page-summary feature. It is moving toward a browser agent. The browser is already where users complete many web tasks. If Gemini can understand pages, fill information, compare options, and execute some steps, Chrome becomes a task execution surface.

The challenge is practical:

Websites are complex, so automated actions can fail.
Forms, payments, logins, and CAPTCHAs require caution.
Users need to know what Gemini did.
Final submission, payment, or booking should usually remain human-confirmed.

The hard part is not only model capability, but browser automation, safety boundaries, and user trust.

Autofill: From Password Filling to Complex Forms

Autofill with Google was mostly about passwords, addresses, and payment details. Google now wants to upgrade it into a smarter form assistant.

With Gemini’s Personal Intelligence, Android can use relevant information from connected apps to fill more complex form fields, including forms in Chrome.

This is very practical. Filling complex forms on mobile is painful: small screen, many fields, and information scattered across email, calendar, chats, and documents. If Gemini can organize and fill this information with user permission, it can save a lot of time.

Google also stresses that connecting Gemini and Autofill with Google is strictly opt-in. Users choose whether to connect them and can turn the connection on or off in settings.

That matters because Autofill touches personal details, addresses, accounts, payments, work information, and sensitive forms. The more useful it becomes, the more important explicit permission and easy opt-out become.

Rambler: Turning Speech into Sendable Text

Rambler is one of the more interesting new features.

Gboard already supports speech-to-text, but natural speech often includes repetition, pauses, filler words, and self-corrections. Rambler’s goal is to turn natural speech into clearer text that is ready to send.

It is useful when:

You want to dictate a message quickly without editing every word.
Your speech includes pauses, repetition, or filler.
You need to turn a rough thought into a more professional text, email, or chat message.
You switch between languages and want the system to understand context.

Google says Rambler will clearly show when it is enabled, and audio is used only for real-time transcription and not saved. This is a response to privacy and transparency concerns.

From a product perspective, Rambler upgrades “voice input” into “voice writing.” It does not only record what you said; it helps turn speech into sendable text.

Natural-Language Widgets

Gemini Intelligence also introduces Create My Widget. Users can describe a widget in natural language, such as “recommend three high-protein meal prep recipes every week,” and Android generates a custom widget for the home screen.

This points toward generative UI. Users no longer pick only from fixed widget templates; they describe the information and presentation they want.

If the idea matures, the phone home screen could become much more personal. Weather, schedule, health, commute, food, learning, and work reminders could all become dynamic modules generated around user needs.

But generative UI also needs stability. A widget is not a one-off chat response. It sits on the home screen for a long time and must be reliable, readable, configurable, and visually controlled.

Material 3 Expressive and Intelligent UI

Google also says Gemini Intelligence will bring design updates based on Material 3 Expressive.

This is not just decoration. When AI starts acting proactively, the UI needs to clearly show:

What the AI is doing.
Which steps are done.
Where user confirmation is needed.
How the user can cancel or change the action.

Proactive AI without clear UI easily makes users feel out of control. Design language becomes part of the AI product experience.

Availability and Rollout

According to Google, Gemini Intelligence features will start on the latest Samsung Galaxy and Google Pixel phones in summer 2026, then expand to more Android devices, including watches, cars, glasses, and laptops.

This is not a global all-at-once launch. Availability may depend on device, region, language, app support, and account settings.

If you want to try it, the realistic expectations are:

Watch Pixel and Samsung flagship phones first.
Watch for system updates after summer 2026.
Look for new toggles in Gemini, Chrome, Gboard, Autofill, and Android settings.
Not every region and language will support every feature at the same time.

What This Means for Android

Gemini Intelligence on Android is not just a bundle of small AI features. It changes Android’s product direction.

Traditional phone operating systems manage apps, notifications, permissions, files, and hardware. Google now wants the system to understand user intent and complete tasks across apps. If this works, Android’s competition will shift from “system features and app ecosystem” toward “how well the system can proactively help users do things.”

It also changes mobile AI competition:

Apple will emphasize on-device integration, privacy, and system-level control.
Google will emphasize Gemini, Search, Chrome, Android, and multi-device ecosystems.
Third-party AI apps will find it harder to compete with system-level entry points.
App developers will need to think about how their apps can be called by AI agents.

In the next few years, AI on phones may no longer be just a chat entry point. It may become the system-level execution layer.

Summary

Gemini Intelligence on Android is not about adding another Gemini chat box to the phone. It puts AI into Android’s operating flow. Multi-step automation, smarter Chrome browsing, Autofill, Rambler, and natural-language widgets all aim to turn the phone from a passive tool into a proactive assistant.

Whether it changes user habits depends on reliability, clear privacy controls, smooth cross-app operation, and keeping users in final control. At least from this announcement, Google is defining the next stage of Android as a proactive AI system, not just a traditional mobile operating system.

Reference:

Google Blog: A smarter, more proactive Android with Gemini Intelligence

Gemini 3.5 Pro Leaks: Google Wants Spark Agent to Win Back the AI Coding Entry Point

Fri, 15 May 2026 23:45:34 +0800

Gemini 3.5 Pro has not been officially released yet, but leaks around it are already heating up.

The current round of information revolves around several keywords: Gemini 3.5 Pro, the codename Cappuccino, Gemini Spark, AI coding, and MCP tool integration. Together, they point in one direction: Google is not just preparing another chat model update. It wants to reconnect models, tools, Agents, and Google ecosystem entry points.

Before an official release, all of this should still be treated as leaked information. The more important signal is not one screenshot or one benchmark claim, but the gaps Google may be trying to close next.

Why Gemini 3.5 Pro Matters

Based on the exposed information, Gemini 3.5 Pro may be a jump in naming.

People were still discussing Gemini 3.2 earlier, and then Gemini 3.5 Pro appeared in leaks. If the naming is real, Google likely wants to tell a bigger version story in the next release rather than ship a routine minor update.

The leaked highlights mainly fall into three areas:

continued improvements in coding and reasoning;
stronger SVG, interactive page, animation, and 3D generation;
a new Agent product, Gemini Spark, potentially moving to the front stage.

None of these directions is surprising. Gemini has long emphasized multimodality, and Google has very strong distribution channels. The real question is whether it can catch up with OpenAI and Anthropic in developer tools and Agent workflows.

Coding Is The Lesson Google Most Needs To Catch Up On

In 2026, coding is no longer just a model benchmark item. It has become one of the most direct product entry points.

The reason is simple: AI coding tools are used frequently and generate a large amount of feedback data. Developers ask models to read code, modify code, run tests, and fix bugs every day. These interactions naturally push the next generation of models and tooling forward.

Over the past year, Claude Code has gained strong mindshare among developers, while OpenAI has kept strengthening the connection between Codex and ChatGPT. Google has products such as Antigravity, but its external presence has not been as strong.

That is why Gemini 3.5 Pro is being watched closely. If it only becomes better at chatting or answering faster, the impact is limited. If it truly improves code understanding, cross-file editing, tool calling, and long-running task execution, it may change developer workflows.

Gemini Spark May Be The Bigger Variable

More aggressive than the model itself is the rumored Gemini Spark.

According to the leaks, Spark is not positioned as a normal chat assistant, but as an always-on AI Agent. It may connect to email, calendars, web pages, tasks, account state, and personal context to help users handle multi-step workflows.

This kind of product has a large imagination space. For example:

automatically organizing an inbox;
following up on tasks for the user;
performing actions on web pages;
handling cross-application workflows;
arranging daily matters based on personal preferences.

But the risks are just as obvious. If an always-on Agent can access login state, browser data, files, location, and third-party services, it must answer several questions: when must the user confirm an action? Which operations must be blocked from automation? Will data be shared with third parties? How are remote browsers and credentials isolated?

So the real question for Spark is not just whether it can get work done. It is whether Google can make permissions, auditing, confirmation flows, and user control clear enough.

What MCP Tool Integration Suggests

The leaks also mention that the new Gemini selector may include MCP-related models or testing entries.

If this ships, it suggests Google is also pushing models from a question-answering system toward a tool operating system. The model will no longer only generate text. It will need to call external tools, access business systems, read and write files, run commands, and maintain task state across multiple steps.

This direction is consistent with OpenAI and Anthropic. Whoever makes tool calling more reliable will have an easier time embedding AI into real workflows.

But MCP integration itself is not the finish line. The hard part is stability:

can the model choose the right tool;
are the parameters reliable;
can it recover after failure;
are permission boundaries clear;
can users trace every step.

If these questions are not solved, more tools also mean a larger surface for mistakes.

Multimodality Is Still Google’s Strong Card

The place where Google has the best chance to differentiate is still multimodality.

Based on exposed SVG, interactive page, animation, and visual generation examples, Gemini may continue to strengthen its ability to generate interactive content from prompts. Compared with simply writing a piece of code, this is closer to product prototyping: the user describes an idea, and the model directly produces an operable, adjustable, previewable interface.

This path fits Google well. It can build on Gemini’s multimodal strengths and also connect with Android, Chrome, Workspace, Search, Ads, and Cloud.

If Google wants to avoid competing only on “whose coding model is stronger”, it may put more emphasis on a more complete multimodal Agent system.

The Three Companies Are Splitting Into Different Playbooks

The current model race is no longer just a leaderboard race.

OpenAI’s advantage lies in product iteration and distribution speed. Codex, ChatGPT, enterprise tools, and APIs are becoming more tightly connected.

Anthropic’s advantage lies in developer mindshare and code model quality. Claude Code has already become the default AI coding entry point for many people.

Google’s advantage is ecosystem access. Gmail, Docs, Chrome, Android, Search, YouTube, Maps, and Cloud services form a huge personal and enterprise data network. If Agents can safely connect to these entry points, Google may move from a “model chaser” to a “workflow entry point controller”.

That is why Gemini Spark is worth watching. It does not necessarily need to rank first on every benchmark. If it enters daily workflows, it may still build its own moat.

How Regular Users Should Read This

For regular users, there is no need to be pulled around by every leak in the short term.

The more practical things to watch are:

Whether Gemini 3.5 Pro’s coding ability truly improves, especially in complex repositories, long context, and tool calling.
Whether Gemini Spark is safe by default, with clear confirmation and traceable records before sensitive operations.
Whether Google gives clear pricing, quotas, and enterprise permission management, rather than only showing demos.

Pretty screenshots alone do not mean much. Whether it can reliably enter real workflows is the dividing line for this round of AI Agent products.

What It Means For Developers

Developers should care less about “which model won” and more about whether their workflow is portable.

Claude Code, Codex, Gemini, Antigravity, Cursor, Windsurf, and many other tools are all competing for the entry point. If every process is locked into one platform, future changes in cost, quota, model policy, or permission rules will make migration painful.

A safer approach is:

keep standard Git workflows for important projects;
always inspect diffs after automated edits;
use tests and CI as backstops for key tasks;
do not hand production credentials to opaque Agents;
when open protocols can connect tools, prefer replaceable options.

Models will keep getting stronger, but engineering discipline will not become obsolete.

Summary

The Gemini 3.5 Pro leaks suggest that Google is accelerating its effort to catch up in AI coding and Agent entry points. Model improvements are only one part of the story; always-on Agents such as Gemini Spark may be the larger strategic move.

But the more a system can “do things automatically” for users, the more it needs strict permission boundaries and verifiable workflows. For Google, the real challenge is not only catching up with GPT-5.5 or Claude. It is combining strong models, safety mechanisms, and ecosystem entry points into a trustworthy daily workflow.

If Google pulls that off, Gemini may not need to top every leaderboard to regain some initiative in AI entry points.

How Can Codex Use Chinese LLMs? Managing OpenAI-Compatible APIs with CCX

Wed, 13 May 2026 23:20:40 +0800

CCX is an AI API proxy and protocol-conversion gateway. It puts Claude Messages, OpenAI Chat Completions, OpenAI Images, Codex Responses, and Gemini API behind one service entry point, while also providing a web management UI for configuring channels, keys, model mappings, priorities, failover, and traffic monitoring.

If you use Claude, OpenAI, Gemini, and Codex at the same time, or maintain multiple upstream services compatible with OpenAI API, CCX is valuable because it gives you one entry point and one management layer. Clients connect to a single service address; CCX decides which upstream channel should handle each request.

Project: https://github.com/BenedictKing/ccx

What problem does CCX solve?

When multiple AI APIs are used together, several problems appear quickly:

Each provider has different paths, authentication, and request formats.
One class of models may have multiple upstreams, requiring manual switching of base URL and API key.
When a key or channel fails, the client usually does not automatically switch to a backup channel.
In team use, it is hard to centrally manage model allowlists, proxies, custom headers, and request logs.
When Claude, Gemini, OpenAI Chat, image APIs, and Codex Responses all need to coexist, configuration becomes scattered.

CCX’s approach is to consolidate these differences into a proxy layer. Frontend tools, scripts, or business services call CCX; CCX then routes the request to a suitable upstream based on API type, model, channel status, priority, and health.

Supported endpoints

CCX exposes one backend entry point. The default port is 3000. Main paths include:

GET  /                         -> Web management UI
GET  /health                   -> Health check
/api/*                         -> Management API
POST /v1/messages              -> Claude Messages proxy
POST /v1/chat/completions      -> OpenAI Chat proxy
POST /v1/responses             -> Codex Responses proxy
POST /v1/images/generations    -> OpenAI Images generation
POST /v1/images/edits          -> OpenAI Images editing
POST /v1/images/variations     -> OpenAI Images variations
GET  /v1/models                -> Model list
POST /v1beta/models/*          -> Gemini proxy

In other words, CCX does not proxy only one protocol. It manages common AI APIs as separate channel types: Messages, Chat, Responses, Gemini, and Images. Different protocols do not share the same health state or log space, which matters when troubleshooting.

Architecture overview

CCX uses a Go backend and Vue 3 frontend. The frontend build is embedded into the backend binary, so it can be deployed on a single port: the same service provides the Web UI, management API, and proxy API.

A request roughly follows this path:

`1`	`Client -> Auth Middleware -> Route Handler -> Channel Scheduler -> Provider / Converter -> Upstream API -> Metrics / Channel Logs -> Client Response`

The main modules can be understood as follows:

handlers: receive requests for different protocols and management operations.
providers: wrap upstream API request and response handling.
converters: handle protocol conversion for scenarios such as Responses.
scheduler: choose channels based on priority, promotion period, health state, circuit breaker state, and trace affinity.
metrics: record request counts, success rate, latency, logs, and circuit breaker state.
config: maintain runtime configuration, with hot reload and backup support.

The design is not about forcing every API into one format. It proxies each protocol type separately, while unifying management, scheduling, logging, and authentication.

CCX vs CodexBridge

CCX and CodexBridge are both related to Codex and OpenAI-compatible APIs, but they solve different problems.

CodexBridge is more like a dedicated Codex bridge. Its main goal is to wrap Codex CLI/SDK as an OpenAI-compatible /v1/chat/completions service, so OpenWebUI, Cherry Studio, scripts, or other OpenAI-compatible clients can call local Codex. In short, CodexBridge focuses on exposing Codex.

CCX is more like a unified AI API gateway. It does not only handle Codex Responses; it also supports Claude Messages, OpenAI Chat, OpenAI Images, and Gemini API, with a web management UI, channel priority, failover, log monitoring, and multi-key management. In short, CCX focuses on managing multiple models and providers together.

Quick comparison:

Item	CodexBridge	CCX
Core positioning	Local Codex bridge	Multi-protocol AI API gateway
Main goal	Turn Codex into an OpenAI-compatible endpoint	Manage Claude, OpenAI, Gemini, Codex, and other channels together
Management UI	Focuses on the API service itself	Provides a web management UI
Multi-channel scheduling	Not the focus	Supports channel priority, failover, and log monitoring
Best fit	Local or single-service Codex calls	Teams, multiple keys, multiple providers, multiple protocols

If you only want to connect Codex to OpenWebUI or Cherry Studio, CodexBridge is more direct. If you want to manage Codex, Claude, Gemini, DeepSeek, Qwen, Kimi, and other upstreams together, CCX is a better fit.

Quick deployment

The simplest way is to download the binary. After downloading it, create .env in the same directory:

PROXY_ACCESS_KEY=your-proxy-access-key
PORT=3000
ENABLE_WEB_UI=true
APP_UI_LANGUAGE=zh-CN

After startup, open:

`1`	`http://localhost:3000`

If localhost does not work from WSL, Docker, PowerShell, or another Windows environment, use the Windows host’s LAN IPv4 address instead, for example:

`1`	`http://192.168.1.23:3000`

By default, CCX listens on :PORT for all network interfaces, so access control matters if it is exposed to a LAN.

Docker deployment

Docker is suitable for long-running service deployment:

docker run -d \
  --name ccx \
  -p 3000:3000 \
  -e PROXY_ACCESS_KEY=your-proxy-access-key \
  -e APP_UI_LANGUAGE=zh-CN \
  -v $(pwd)/.config:/app/.config \
  crpi-i19l8zl0ugidq97v.cn-hangzhou.personal.cr.aliyuncs.com/bene/ccx:latest

If the repository already has docker-compose.yml, you can also run:

`1`	`docker compose up -d`

For automatic updates, add the Watchtower configuration:

`1`	`docker compose -f docker-compose.yml -f docker-compose.watchtower.yml up -d`

After deployment, .config stores runtime configuration and persistent data. Mount it to the host to avoid losing configuration when the container is recreated.

Running from source

For development or custom builds:

git clone https://github.com/BenedictKing/ccx
cd ccx
cp backend-go/.env.example backend-go/.env
make run

Common commands:

make dev
make run
make build
make frontend-dev

Frontend-only development:

1
2
3

cd frontend
bun install
bun run dev

Backend-only development:

1
2

cd backend-go
make dev

Key environment variables

Minimal usable configuration usually includes:

PORT=3000
ENV=production
ENABLE_WEB_UI=true
PROXY_ACCESS_KEY=your-proxy-access-key
ADMIN_ACCESS_KEY=your-admin-secret-key
APP_UI_LANGUAGE=zh-CN
LOG_LEVEL=info
REQUEST_TIMEOUT=300000

Notes:

PROXY_ACCESS_KEY is used for the proxy API and must be changed.
ADMIN_ACCESS_KEY is used for the Web UI and /api/*; it should be separate from the proxy key.
ENABLE_WEB_UI controls whether the management UI is enabled.
REQUEST_TIMEOUT controls request timeout; increase it for long-context or image tasks.
LOG_LEVEL controls log verbosity; production usually uses info or warn.

To limit request body size, check:

`1`	`MAX_REQUEST_BODY_SIZE_MB=50`

Image editing, base64 images, and multimodal requests can all increase request body size.

Channel orchestration and failover

The CCX management UI can configure multiple channels, with options such as:

Upstream service type.
API key or multi-key rotation.
Proxy address.
Custom request headers.
Model allowlist.
Route prefix.
Priority.
Health checks and circuit-breaker recovery.

Scheduling considers channel state, priority, promotion period, trace affinity, circuit-breaker state, and available keys. In simple terms:

Under normal conditions, higher-priority channels are used first.
If one channel fails, CCX can fail over to a backup channel.
Circuit breaking avoids repeatedly hitting an obviously unavailable upstream.
Trace affinity tries to keep related sessions on suitable channels.

These features are useful when you have multiple keys, providers, or regional upstreams. For personal lightweight use, you can also configure only one channel and use CCX as a proxy layer with a Web UI.

Logs and monitoring

CCX provides channel metrics and request logs, including:

Request volume.
Success rate.
Failure rate.
Average latency.
Historical data by model.
Channel status and circuit-breaker state.

For production, use relatively conservative logging:

ENV=production
LOG_LEVEL=info
ENABLE_REQUEST_LOGS=true
ENABLE_RESPONSE_LOGS=false

This keeps basic request information while avoiding full response content in logs. You can temporarily enable more detailed logs for troubleshooting, but restore the safer configuration afterward, especially in production.

Security recommendations

CCX is a proxy gateway and stores upstream API keys, so deployment should not stop at “it runs.” At minimum:

Do not use a default or short PROXY_ACCESS_KEY.
Set a separate ADMIN_ACCESS_KEY.
Do not expose the Web UI directly to the public internet.
If public access is required, place it behind a reverse proxy, VPN, access control, or SSO.
Do not commit .env, .config, or log files to Git.
Do not keep full request and response body logging enabled in production.

You can generate random keys like this:

1
2

PROXY_ACCESS_KEY=$(openssl rand -base64 32)
ADMIN_ACCESS_KEY=$(openssl rand -base64 32)

Who should use it?

CCX is better suited to these scenarios:

Maintaining Claude, OpenAI, Gemini, Codex, or image APIs at the same time.
Having multiple API keys that need rotation, routing, and failover.
Managing upstream channels through a Web UI instead of editing config files manually.
Observing success rate, latency, and logs for each channel.
Providing one unified AI API entry point for a team.

If you only call one model occasionally on your own machine, the official SDK or a single OpenAI-compatible proxy is simpler. CCX’s advantage is multi-channel, multi-protocol, unified operation.

Summary

CCX is an AI API gateway, not a client for one specific model. It puts Claude Messages, OpenAI Chat, OpenAI Images, Codex Responses, and Gemini into one proxy layer, with channel orchestration, failover, logs, monitoring, and a Web management UI.

For individuals, it reduces the trouble of switching API addresses and keys. For teams or long-running services, it is closer to a lightweight AI gateway. Before production use, the important work is not only configuring models, but also securing keys, the management entry point, logging levels, channel priority, and failover strategy.

References

GitHub project: https://github.com/BenedictKing/ccx
Architecture notes: https://github.com/BenedictKing/ccx/blob/main/ARCHITECTURE.md
Environment variables: https://github.com/BenedictKing/ccx/blob/main/ENVIRONMENT.md

How ChatGPT, Claude Code, and Gemini memory mechanisms differ

Thu, 07 May 2026 14:47:17 +0800

“Memory” is becoming increasingly important in AI products. It marks the shift from one-off conversations to long-term collaboration: you do not need to reintroduce your background, repeat your preferences, or ask the model to understand the same project again and again.

But memory does not mean the same thing in every product. ChatGPT, Claude Code, and Gemini all try to help AI remember longer, but their goals, storage locations, transparency, and use cases are very different.

As of May 7, 2026, they can be roughly understood as three types:

ChatGPT is more like personal assistant memory.
Claude Code is more like engineering project memory.
Gemini is more like Google ecosystem context.

ChatGPT: long-term preferences around the person

ChatGPT memory is mainly designed for personal collaboration. It cares about who you are, what you prefer, and what you work on over time.

OpenAI currently separates ChatGPT memory into saved memories and chat history.

saved memories are important pieces of information ChatGPT stores, such as your name, preferences, goals, common tech stack, and writing habits. You can explicitly ask it to remember something, and it may also save information from conversation when it thinks it will be useful later.

chat history lets ChatGPT reference past conversations when answering. It does not mean every chat becomes a permanent memory. Instead, ChatGPT can search past conversations for relevant context when needed.

So ChatGPT’s core logic is: understand the same user across sessions.

Typical examples include:

“Keep code examples concise for me.”
“I mainly use Python and TypeScript.”
“I am writing a Hugo blog about AI tools.”
“I prefer conclusions first, then details.”

These memories are not bound to one project. They follow the account and the user’s working habits.

Memory Sources: making personalization more visible

OpenAI emphasized Memory sources in its May 2026 update.

The purpose is not to add another type of memory, but to show users what sources ChatGPT referenced when personalizing a response. According to OpenAI help documents, Memory Sources may show:

Past chats.
Saved memories.
Custom instructions.
Files in the file library.
Emails from connected Gmail.

Files and Gmail visibility depend on plan, region, and connection status. OpenAI also states that Memory sources may not show every factor that influenced a response, but they help users understand and manage personalization.

This matters. The more AI can “remember you,” the more users need to know what it used to answer. Otherwise personalization becomes a black box: it seems to know you, but you do not know why.

ChatGPT’s advantage is cross-session, cross-topic understanding of personal preferences. The risk is that memories can become outdated, or users may forget an old memory is still affecting answers. It is worth periodically cleaning saved memories and old chats.

Claude Code: around codebases and engineering rules

Claude Code memory is more engineering-oriented. It cares less about a user’s everyday preferences and more about how this codebase should be changed.

Claude Code has two memory mechanisms that are easy to confuse:

Explicit project memory: CLAUDE.md.
Automatic project memory: Auto Memory.

CLAUDE.md is the most basic and stable project memory file. It can live at the project root or inside subdirectories. Claude Code reads these files as project instructions and operating rules.

Good content for CLAUDE.md includes:

Common build, test, and lint commands.
Code style and naming rules.
Project architecture notes.
Module boundaries and risky areas.
Team conventions and commit workflow.

If CLAUDE.md is stored in the repository, it can be committed to Git and shared as a team agent guide. This is completely different from ChatGPT’s cloud-based personal memory.

Claude Code Auto Memory: accumulating project experience

Claude Code also has Auto Memory. Its goal is to let Claude automatically accumulate project knowledge across sessions without requiring users to write every note manually.

According to Claude Code documentation, Auto Memory lets Claude save notes while working, such as build commands, debugging discoveries, architecture notes, code style preferences, and workflow habits. It does not save every session, but judges what may be useful later.

One common misconception is that Auto Memory writes by default to .claude/memory.md in the project root. Official documentation says each project has its own memory directory under the user’s home directory, with a path like:

`1`	`~/.claude/projects/<project>/memory/`

MEMORY.md loads the first 200 lines or 25KB at the start of each conversation, while detailed content may be split into other topic files. Auto Memory files are local Markdown files, and users can view, edit, or delete them through /memory.

This makes Claude Code memory more like a local project knowledge base. It is closer to the codebase than ChatGPT’s personal memory, and more dynamic than a plain CLAUDE.md.

But Auto Memory is local to the machine. It does not naturally follow the repository to other machines or cloud environments. For team-shared stable rules, put them in the repository’s CLAUDE.md.

Gemini: around Google ecosystem context

Gemini’s memory logic is different again.

Gemini also supports saved information and past-chat references. Google help documents say users can save information about life, work, or preferences, and Gemini can reference past chats before answering. When it uses this information, the response may show sources such as Your saved info or Previous chats.

But Gemini’s differentiation is not only “saving a few preferences.” It is Google ecosystem integration.

With user authorization and feature availability, Gemini can access context from connected Google apps such as Gmail, Google Drive, Docs, and Sheets. Its advantage is not making users teach it every item manually, but turning existing Google account data into searchable work context.

A typical difference:

ChatGPT remembers: “I have been repairing an LTO tape drive recently.”
Gemini may find the purchase confirmation email in Gmail or read repair notes from Drive.

This does not mean Gemini can unconditionally read all Google data. It depends on account type, region, permissions, connected apps, Keep Activity settings, and product availability. Enterprise and school accounts may also be controlled by Google Workspace administrators.

More accurately, Gemini memory is a combination of saved info, past chats, and connected Google ecosystem data.

Core differences

Dimension	ChatGPT	Claude Code	Gemini
Core object	Person and preferences	Project and codebase	Google account and ecosystem data
Typical memory	Preferences, background, long-term goals	Architecture, commands, conventions, debugging experience	Saved info, past chats, Gmail/Drive/Docs context
Storage form	Memory and chat context in OpenAI account	`CLAUDE.md`, `MEMORY.md`, local Markdown files	Google account activity, saved info, connected app data
Transparency	Memory sources show part of the source	Markdown files can be opened and edited	Managed through source prompts, Gemini Apps Activity, and Google settings
Cross-project ability	Strong, follows user account	Weak, mainly follows project or local project memory	Strong, depends on Google data and permissions
Team sharing	Not suitable for direct sharing	`CLAUDE.md` can be shared through Git	Mainly depends on Workspace and permissions
Best for	Personal preferences and long-term assistant behavior	Long-term coding projects and agent collaboration	Google Workspace retrieval and cross-tool work

How to choose

If you want AI to remember who you are, what style you prefer, and how you usually work, ChatGPT memory is more suitable.

It is good for saving personal preferences such as writing style, tech stack, answer format, professional background, and long-term project direction. Its focus is reducing self-introduction cost so each new conversation can start faster.

If you want AI to remember how a codebase should be changed, which commands work, and which traps to avoid, Claude Code is more suitable.

Put stable rules into CLAUDE.md for team sharing. Let Auto Memory assist with dynamic experience. Important decisions should still be organized into documentation or CLAUDE.md, not left only in local automatic memory.

If most of your materials live in Gmail, Drive, Docs, and Sheets, Gemini’s ecosystem context has an advantage.

It is useful for finding old emails, organizing Drive documents, and connecting calendar and office materials. The key to using Gemini is not repeatedly reminding it in chat, but making sure the relevant app connections, permissions, and activity settings are correct.

A practical division of labor

You can divide them like this:

ChatGPT remembers general personal preferences.
Claude Code remembers engineering knowledge for a repository.
Gemini retrieves materials from your Google ecosystem.

In other words, ChatGPT is like a personal secretary, Claude Code is like a senior engineer inside the project, and Gemini is like an indexer for your Google account.

There is no absolute winner. They have different goals.

The biggest mistake is mixing them together. Personal preferences do not always belong in project memory; project architecture does not always belong in cloud personal memory; and Google ecosystem retrieval does not mean the model has truly understood you long-term.

Short Take

The next stage of AI memory is not simply “remember more.” Memory needs layers, visibility, and control.

ChatGPT focuses on cross-session personalization. Claude Code focuses on code project continuity. Gemini focuses on Google ecosystem context. Good long-term AI collaboration does not put all information into one black box; it keeps different kinds of memory in the right places.

Put personal preferences in personal memory, engineering rules in the codebase, and historical materials in the original document and email systems. AI’s job is to call the right context when needed, not mix everything into one pile.

Gemini Embedding 2: Putting Text, Images, Video, and Audio in One Vector Space

Mon, 04 May 2026 06:01:10 +0800

Google Developers Blog introduced how to build with Gemini Embedding 2. The model is now generally available through the Gemini API and Gemini Enterprise Agent Platform. The important point is not simply that it is a new embedding model, but that it maps text, images, video, audio, and documents into the same semantic space.

This broadens what retrieval systems can handle. Many RAG pipelines previously had to convert images, video, or audio into text or metadata before indexing them separately. Gemini Embedding 2 can process multimodal inputs directly, making it easier for agents, search systems, and classifiers to work with real business materials.

Original article: Building with Gemini Embedding 2: Agentic multimodal RAG and beyond

Model Capabilities

Gemini Embedding 2 supports more than 100 languages. A single request can process:

Up to 8,192 text tokens
Up to 6 images
Up to 120 seconds of video
Up to 180 seconds of audio
Up to 6 pages of PDF

Its key idea is a unified semantic space. Developers can place content from different modalities into one vector representation system, then use the same retrieval, clustering, or reranking logic to process it.

For example, a text description and an image can be included in the same embedding request:

from google import genai
from google.genai import types

client = genai.Client()

with open('dog.png', 'rb') as f:
    image_bytes = f.read()
result = client.models.embed_content(
    model='gemini-embedding-2',
    contents=[
        "An image of a dog",
        types.Part.from_bytes(
            data=image_bytes,
            mime_type='image/png',
        ),
    ]
)

print(result.embeddings)

If you want a separate embedding for each input rather than one aggregated vector, you can use the Batch API. The original article also notes that Agent Platform support for this kind of batch workflow is still in progress.

What It Means for RAG

Multimodal embeddings are useful for agentic RAG. An AI agent may need to inspect a code repository, PDFs, screenshots, charts, audio meeting notes, and product images at the same time. If all of these materials can enter the same semantic space, the retrieval pipeline no longer needs a separate entry point for every format.

Google recommends using task prefixes according to the goal of the task, so the embeddings better match the retrieval objective. For example, question answering, fact checking, code retrieval, and search results can use different prefixes:

# Generate embedding for your task's query:
def prepare_query(query):
    return f"task: question answering | query: {content}"
    # return f"task: fact checking | query: {content}"
    # return f"task: code retrieval | query: {content}"
    # return f"task: search result | query: {content}"
# Generate embedding for document of an asymmetric retrieval task:
def prepare_document(content, title=None):
    if title is None:
        title = "none"
    return f"title: {title} | text: {content}"

This kind of prefix is suitable for asymmetric retrieval: user queries are often short, while documents are often long. Formatting query and document differently for the task can improve matching between short queries and long documents.

The original article gives two real-world examples:

Harvey saw a 3% increase in Recall@20 precision on legal retrieval benchmarks compared with its previous embeddings.
Supermemory saw a 40% increase in Recall@1 search accuracy and uses it across memory, indexing, search, and Q&A pipelines.

These numbers do not mean every scenario will improve by the same amount, but they show that multimodal embeddings are already producing results in real retrieval products, not only demos.

Visual Search

Gemini Embedding 2 is also suitable for image-to-image search, image-text hybrid search, and product identification. The original article mentions Nuuly, URBN’s clothing rental company, using it to match photos of untagged garments in warehouses against its catalog. Match@20 improved from 60% to nearly 87%, and the overall successful identification rate rose from 74% to over 90%.

The point in this type of scenario is not content generation, but understanding which inventory item, document, or product record is closest to a given image. If your business has many images, video clips, or scanned documents, multimodal embeddings can be more natural than text-only indexing.

Search Reranking

Embeddings can also be used for reranking. A common approach is to first retrieve a set of candidate results, then calculate the similarity between each candidate and the user’s query, pushing more relevant content to the top:

# 1. Define a function to calculate the dot product (cosine similarity)
def dot_product(a: np.ndarray, b: np.ndarray):
  return (np.array(a) @ np.array(b).T)
# 2. Retrieve your embeddings
# (Assuming 'summaries' is your list of search results)
search_res = get_embeddings(summaries)
embedded_query = get_embeddings([query])

# 3. Calculate similarity scores
sim_value = dot_product(search_res, embedded_query)

# 4. Select the most relevant result
best_match_index = np.argmax(sim_value)

The original article also mentions another idea: first ask the model to generate a baseline hypothetical answer from its internal knowledge, embed that answer, and compare it with candidate content to find the most semantically relevant result. This is especially useful for Q&A-style RAG.

Clustering, Classification, and Anomaly Detection

Beyond retrieval, embeddings are also useful for clustering, classification, and anomaly detection. Unlike the asymmetric question-answering retrieval above, these are symmetric tasks, where the same task prefix can be used for both query and document:

# Generate embedding for query & document of your task.
def prepare_query_and_document(content):
    # return f'task: clustering | query: {content}'
    # return f'task: sentence similarity | query: {content}'
    # return f'task: classification | query: {content}'

These tasks can be used for sentiment classification, content moderation, similar asset grouping, and anomaly discovery. They can also help agents organize large amounts of context before moving into later reasoning steps.

Storage and Cost

Gemini Embedding 2 outputs 3,072-dimensional vectors by default. It uses Matryoshka Representation Learning, so vectors can be truncated to smaller dimensions with output_dimensionality. Google recommends 1,536 or 768 dimensions when efficiency is the priority:

result = client.models.embed_content(
    model="gemini-embedding-2",
    contents="What is the meaning of life?",
    config={"output_dimensionality": 768}
)

Vectors can be stored in Agent Platform Vector Search, Pinecone, Weaviate, Qdrant, ChromaDB, and similar systems. For cost, the original article notes that the Batch API provides higher throughput and reaches 50% of the default embedding price.

How Developers Can Use It

If you already have text-based RAG, you can start with two incremental upgrades:

Put PDFs, screenshots, image descriptions, and text documents into the same index, then test whether retrieval recall becomes more stable.
Add task prefixes for different tasks, such as question answering, fact checking, code retrieval, and product search. Do not process all content with the same embedding format.

If you are building a new product, consider these directions first:

Enterprise knowledge bases: retrieve documents, charts, presentation screenshots, and meeting materials together.
Visual search: use images, text, or mixed inputs to find products, assets, design drafts, and archives.
Agent toolchains: let coding agents, research agents, or customer support agents retrieve business materials in multiple formats.
Content governance: classify, cluster, and detect anomalies across text, images, and video clips.

The value of Gemini Embedding 2 is that it turns multimodal materials into one searchable asset system. For developers, this reduces the need for an intermediate “convert to text, then retrieve” layer and makes RAG systems closer to the shape of real-world data.

Gemini Comes to Cars with Google Built-in: In-Car Voice Assistants Start to Feel More Like Real AI Assistants

Fri, 01 May 2026 06:09:57 +0800

Google announced on April 30, 2026 that Gemini is starting to roll out to cars with Google built-in, as an upgraded version of Google Assistant.

The point is not simply that cars are getting another AI assistant. It is that in-car voice interaction is moving from fixed commands toward more natural, continuous conversation. Users no longer need to remember rigid command formats. They can speak more naturally and ask Gemini to help with navigation, messages, vehicle information, and even some in-car settings.

Starting with English users in the United States

According to Google, this update will cover both new and existing vehicles, as long as the vehicle supports Google built-in and the user is signed in to their Google account in the car.

The rollout will begin with English-language users in the United States, then expand to more languages and countries. Eligible users will see an option in the car to upgrade to Gemini. After upgrading, they can call up Gemini in several ways:

Say Hey Google
Tap the microphone on the home screen
Use the voice button on the steering wheel

This shows that Google is not turning Gemini into a new entry point that users have to learn from scratch. It keeps the existing in-car voice entry point, while replacing the underlying assistant with a stronger Gemini experience.

In-car voice no longer depends only on fixed commands

A common problem with traditional in-car voice assistants is that they can do quite a lot, but users have to speak in a very “standard” way. As soon as the request becomes a little complex, the assistant may fail to understand it or only perform a basic action.

With Gemini in the car, Google is emphasizing natural conversation. For example, a user can simply say:

I need to grab lunch, find some highly rated sit-down restaurants along the way. I’m not in a rush, oh, and I’d like to eat outside.

Gemini can use Google Maps information to find suitable restaurants along the route. The user can then follow up by asking about parking or vegetarian options, without starting a whole new search.

This interaction fits the driving context better. When driving, it is hard to repeatedly filter, tap, and revise options as you would on a phone. If a voice assistant can understand more complete intent, it can noticeably reduce distraction.

Maps, messages, and music become easier to handle

The examples Google gives are mostly built around the most common needs while driving.

The first category is route and place search.

Gemini can use Google Maps information to find restaurants, attractions, or charging stations along the way, and it can also answer questions related to the current route. For example, when passing near a stadium, the user can ask whether there is an event nearby and whether it will affect traffic.

The second category is message handling.

Users can ask Gemini to summarize new text messages and then reply based on the context. For example, they can ask it to tell a friend “I’m on my way” and include the estimated arrival time. If they want to change the message, they can add more instructions without starting over.

The third category is music and ambience.

Users do not necessarily need to know the name of a radio station or a specific playlist. They can simply describe what they want to hear. For example, they can ask for a jazz radio station, or ask YouTube Music to play upbeat ’70s folk-rock for a mountain drive while skipping slow ballads.

These functions are not entirely new by themselves. Gemini’s value is in handling multiple conditions in a single natural-language request, instead of forcing users back into fixed commands.

Gemini Live lets people keep talking while driving

Google also mentioned that Gemini Live will enter the in-car experience and is currently in beta. Users can tap the Gemini Live button or say Hey Google, let's talk to start a more free-flowing conversation.

This scenario is closer to learning and brainstorming while driving. For example, when driving to Lake Tahoe, users can ask Gemini to share local history and fun facts. If something sounds interesting, they can interrupt and ask follow-up questions. Gemini can also help plan hikes and activities after arrival.

The difference from traditional in-car assistants is clear. A traditional assistant is more like a tool button; Gemini Live is more like a voice interface that supports continuous conversation.

Owner’s manuals and real-time vehicle status are the key differences

More importantly, Gemini does not only answer general questions. Google says it has worked with automakers to integrate Gemini more deeply with vehicle systems.

This brings several capabilities closer to the car itself.

First, users can ask about vehicle features.

For example: “How should I prepare my car for an automatic car wash?” or “My garage ceiling is low and the trunk is hitting it. How do I program the trunk so it doesn’t open all the way?” Gemini can answer based on manufacturer-provided owner’s manuals, tailored to the specific vehicle model. The availability and level of detail will vary by brand and model.

Second, EV users can ask about real-time battery level and range.

For example, they can ask for the current battery level, the estimated battery level on arrival, or ask Gemini to find nearby charging stations. Gemini can also combine this with Google Maps and help find nearby cafes while charging.

Third, some in-car settings can be adjusted through natural language.

Google’s example is a user saying that the car is foggy and freezing. Gemini can understand the intent, turn up the heat, and switch on the defroster.

These capabilities are more practical than simply moving a chatbot into the dashboard. A car is an environment with clear state, hardware capabilities, and safety boundaries. If an AI assistant can understand vehicle context, its value is much higher than ordinary Q&A.

The boundaries of in-car AI matter even more

The requirements for AI in a car are different from those on a phone or web page.

When driving, users cannot keep looking at the screen or spend much attention correcting AI. The assistant needs to be concise, reliable, and avoid creating new burdens in critical situations.

So Gemini entering cars does not mean every complex task belongs in the car. A more reasonable direction is:

Reduce the operation cost of navigation and information lookup
Replace multi-level menus with natural language
Help users quickly understand vehicle features
Handle messages and media without increasing distraction
Give EV users smoother charging and route information

On the other hand, high-risk operations still need clear boundaries. Actions that affect driving safety, messages that require confirmation, and vehicle-control operations should all have sufficiently explicit confirmation flows.

Conclusion

Gemini coming to cars with Google built-in is another step in AI assistants expanding from phones and web pages into everyday environments.

Its significance is not that people can finally “chat” in the car. It is that in-car voice assistants are beginning to understand more complex intent and combine maps, messages, music, owner’s manuals, and some vehicle-status information to complete tasks.

If the rollout goes well, in-car voice interaction may gradually move from “remember the command” to “describe the need.” That matters for driving, because a genuinely good in-car AI should not require the driver to give it too much attention.

Reference link:

Your car with Google built-in is about to get smarter, thanks to Gemini - Google Blog

How to Split Tasks Between ChatGPT, Claude, and Gemini: Choosing for Daily Use, Coding, and Special Capabilities

Sat, 25 Apr 2026 10:51:19 +0800

Many people no longer rely on just one model. Instead, they switch back and forth between ChatGPT, Claude, and Gemini. That makes the question much more practical: which kinds of tasks should go to which model?

This feels confusing not because all three are weak, but because they are now strong in different ways. If you still choose based on a vague standard like “which one is smarter,” you can easily end up picking the wrong tool.

If we simplify the conclusion first, it roughly looks like this:

For daily conversations and general-purpose tasks, many people start with ChatGPT
For command-line coding, long-context collaboration, and sustained task execution, Claude often feels smoother
When you need Google ecosystem integration, search, multimodal entry points, or certain product-level capabilities, Gemini tends to stand out more

Let’s break that down into three parts.

1. Daily conversations: why many people still open `ChatGPT` first

For most everyday scenarios, ChatGPT still feels like the “default entry point.”

This is not about a single benchmark. It is about the overall experience:
when you want to ask a quick question, organize your thoughts, draft some copy, create a first version, or summarize a piece of material, ChatGPT usually feels fairly balanced.

Its strengths often show up in a few places:

Its response style is relatively stable
The learning curve is low for general users
Most broad tasks do not require much extra prompt tuning
The product feels polished and works well for frequent everyday use

So if your task is something like this:

Help me organize a topic
Turn an idea into structured content
Summarize a long article
Brainstorm several approaches
Rewrite something more clearly

Then ChatGPT is often a very natural place to start.

That does not mean it is always the strongest option for every professional task. It means that for broad, general-purpose use, it often feels more like the default workspace.

2. Command-line coding and long tasks: why many people lean toward `Claude`

Once a task shifts from “let’s chat” to “let’s keep working until this is done,” many people start preferring Claude.

This is especially true in scenarios like:

Command-line programming
Understanding the context of a large project
Coordinating edits across multiple files
Debugging long task chains
Reading code while steadily moving a task forward

In this kind of work, the key is usually not whether one reply is especially impressive. It is whether the model can stay stable across a longer chain of work.

The reason Claude is often favored is usually not that “it says one sentence better than the others,” but that:

It holds up better on long-context tasks
It feels steadier when reading files, logs, and rules continuously
It is better suited to gradually advancing complex coding work
In command-line and agent workflows, it is often treated as the primary working model

If you are doing vibe coding, fixing bugs in the terminal, understanding project structure, or changing features across multiple files, Claude’s strengths tend to show up more clearly.

Put simply, Claude feels more like a model you work with to get things done, not just one you ask a question and get an answer from.

3. `Gemini` often wins not by “competing head-on in everything”

When people talk about Gemini, they often frame the question like this: is it the strongest of the three?

But in real usage, the more useful question is usually not that. It is: in which scenarios is it especially worth pulling out and using on purpose?

Gemini’s value often shows up more clearly in these directions:

Integration with the Google ecosystem
Search and information gathering
Multimodal entry points
Certain product-side feature linkages

If your workflow is already close to Google’s toolchain, for example:

Search
Documents
Email
Browser-side usage
Mobile entry points

Then Gemini’s practical convenience may matter more than a simple model-score comparison.

In other words, Gemini is often useful because it plugs into your workflow more naturally, not just because it may or may not beat someone else in a single response.

4. The useful way to choose is not asking who is strongest, but asking what kind of task you have

When people compare all three models side by side, the easiest trap is trying to find one “single best” model.

But real tasks vary too much:

Some are one-off Q&A
Some are long-running conversations
Some are software projects
Some are information retrieval
Some are multimodal processing
Some are toolchain collaboration

So the more effective approach is usually to sort by task type:

If you want a broad, high-frequency assistant that works right away, start with ChatGPT
If you need long context, command-line work, coding collaboration, and steady progress on complex tasks, try Claude first
If you need help from the Google ecosystem, search, multimodal entry points, or certain product integrations, pay special attention to Gemini

That kind of division of labor is much closer to real-world use than forcing a single overall champion.

From a light user’s perspective, paying for all three can look redundant.
From a heavy user’s perspective, it is more like assigning different tools to different jobs.

The reason is simple:
if the strengths of the three models have already started to diverge clearly, then using them together is not really duplicated spending. It is a way to reduce switching costs and trial-and-error costs.

For example:

Use ChatGPT for daily organization and general Q&A
Use Claude for primary coding work
Use Gemini for certain search, multimodal, or Google-related workflows

The logic of this setup is not fundamentally different from designers installing multiple creative tools or developers using multiple IDEs.

6. When you should not switch models too often

Of course, having more models is not always better.

If you are still building a stable workflow, jumping too early and too often between three models can actually make things messier. Common issues include:

Re-explaining the same task three times
Getting different suggestions from different models and struggling more to judge them
Losing context and increasing collaboration costs
Getting stuck on tool choice before forming your own working boundaries

So a steadier way is usually this:

Give each model one primary scenario first
Use it continuously in that scenario for a while
Gradually build your own habits of division of labor

That makes it easier to gain reusable experience instead of staying forever in the “let me try this one today” stage.

7. A simple way to remember it

If you just want a practical version to remember, you can use this plain-language split:

ChatGPT: more like the default general-purpose assistant
Claude: more like the main option for long tasks and coding collaboration
Gemini: more like the tool with stronger advantages in search, multimodal work, and the Google ecosystem

This is not an absolute rule, and it does not mean the three cannot replace each other. It is simply a more realistic starting point.

What really matters is not choosing the “strongest model in the universe,” but figuring out as soon as possible:
for the kind of task in front of you, which model saves the most time, costs the least mental effort, and makes it easiest to get results?

Google App for Desktop: Bringing AI Search to Windows

Sat, 18 Apr 2026 11:08:00 +0800

Google has added a lighter search entry point to the Windows desktop. Instead of opening a browser first, you can press a shortcut, bring up a search box, ask questions, upload images, analyze files, select content on your screen, and keep asking follow-up questions.

The official name is Google app for desktop. It is not meant to replace a traditional browser. Its purpose is to bring Google Search, AI Mode, Google Lens, screen sharing, computer file search, and Google Drive search into one desktop search box.

If you often look up information, summarize documents, identify screenshot content, or want a faster way to search your computer, this desktop Google App is worth trying.

Requirements

According to Google’s official page, the current requirements are:

Users must be 13 or older.
The device must run Windows 10 or later.
The app is currently only available in English.
AI Mode in Google Search is not supported for all accounts, countries, and languages.

In other words, Windows 10 and Windows 11 users can install it first and test it. The official page currently says Now available on Windows, so this article focuses on the Windows version.

Main Features

1. Open Search With a Shortcut

After installation, press:

`1`	`Alt + Space`

This opens the Google desktop search box. Press it again to hide the box.

The experience is a bit like a system launcher. Whether you are writing a document, reading a web page, organizing files, or using another app, you can call up Search without switching back to the browser.

2. AI Mode and Follow-Up Questions

Traditional search usually gives you a list of links. AI Mode is closer to a summarized answer built from search results. You can ask a question directly, get a more complete response, and still open helpful links when needed.

The useful part is that you can keep asking follow-up questions. For example, start with:

`1`	`What is this tool best used for?`

Then continue with:

`1`	`If I create written content, how can I use it to improve my workflow?`

This avoids repeatedly rewriting keywords or jumping between multiple pages.

3. Upload Images for Recognition and Search

The desktop Google App supports uploading an image and asking questions about it. Common uses include:

Identifying people, places, products, or objects in an image.
Finding similar images and related information.
Extracting a description from image content.
Asking AI to generate creative prompts based on the image.

For example, after uploading a portrait, you could ask:

`1`	`Who is the person in this image? Please provide an introduction and related references.`

For everyday image lookup, source finding, and object recognition, this is more convenient than manually opening a web page and uploading the image there.

4. Select Screen Content With Google Lens

Google Lens is one of the most useful parts of the desktop app. You can select an area on your screen and let it recognize and search that content.

It is useful for:

Selecting a product on a web page to find similar items or related information.
Selecting text in a screenshot and asking for an explanation.
Selecting a software interface and asking what tool it is.
Selecting an error message and asking for troubleshooting ideas.

The core idea is simple: search what you see. Previously, you might have needed to take a screenshot, save it, and upload it. Now you can select the target directly on the screen.

In addition to selecting one part of the screen, the app also supports screen sharing. Once enabled, AI can see the current window or the entire screen, and you can ask questions about what is visible.

For example, while reading an article, you can ask:

`1`	`Please summarize the key points on the current page.`

Or:

`1`	`What parts of this page could be improved?`

This can save time when reading web pages, reviewing designs, analyzing code snippets, or summarizing long pages. When screen sharing is enabled, the system usually displays a clear border so you can confirm what is being shared.

6. Search Computer Files and Google Drive

Google’s official page also says the app can find apps and files across your computer and Google Drive from the same search box.

This combines desktop search and cloud search. If you remember part of a file name, a keyword from the content, or want to quickly find something in Google Drive, you do not need to open File Explorer and Drive separately.

On first use, the app may ask whether you want to enable Google Drive search or local file search. Authorize only the areas you actually need, especially when local files and cloud data are involved.

Installation and Usage

1. Open the Official Download Page

Visit:

https://search.google/google-app/desktop/

Click Download app to download the installer.

2. Install the Desktop App

After the download finishes, run the installer and follow the prompts.

You can choose to sign in with a Google account. Signing in is more useful if you want Google Drive search, personalized search, and some AI features. If you only want to try basic search, you can also start with the default prompts.

Press:

`1`	`Alt + Space`

A Google search box appears on the desktop. You can type a question directly, upload files, use Lens, or start screen sharing.

4. Enable the Search Scope You Need

If you want to search Google Drive or local files, follow the prompts to enable the relevant permissions.

A practical approach:

Enable Google Drive search first and test cloud document search.
Enable local file search only if you need it.
Leave unnecessary scopes disabled.

This gives you access to the useful parts while keeping permissions clear.

Common Use Cases

Analyze PDFs and Documents

You can drag in a PDF, table, or document and ask it to summarize the key points.

For example:

`1`	`Please summarize the key points of this PDF and list anything I should pay attention to.`

For dense files such as forms, statements, application materials, or manuals, continue with:

`1`	`Please organize the key information by category.`

This is usually faster than reading a long document page by page.

Summarize Web Pages

After enabling screen sharing, you can ask it to summarize the current page.

For example:

`1`	`Extract the main ideas from this page and summarize them in five bullet points.`

This is useful for long articles, product pages, documentation, and news pages.

Identify Screenshots and Interfaces

Use Google Lens to select a software interface, code snippet, error message, or image content on the screen, then ask it to explain what you are seeing.

For example:

`1`	`Please explain what this error means and suggest a troubleshooting path.`

Or:

`1`	`What tool is shown in this screenshot, and what scenarios is it likely used for?`

Help With Content Creation

It can also help write titles, generate outlines, and organize selling points.

For example:

`1`	`Write 10 article titles about AI tool recommendations, with angles for practical tips, productivity tools, and office workflows.`

After getting a draft, you can continue:

`1`	`Make these titles more suitable for a tech blog.`

This follow-up style feels more natural than asking a search engine for everything in one shot.

Usage Tips

If you only need daily information lookup, treat it as a faster Google Search entry point. If you often work with images, PDFs, web pages, and screenshots, focus on Lens, file uploads, and screen sharing.

Keep three points in mind:

The official page says the app is currently only available in English. Chinese prompts may behave differently depending on your account and region.
AI Mode is not available to every account. If you cannot see it, check your account, region, and language settings.
Local file search, Google Drive search, and screen sharing involve privacy permissions. Confirm what content you want the app to access before enabling them.

Quick Summary

The main value of Google app for desktop is that it moves search out of the browser and turns it into an AI search box you can call up at any time.

In short, it can:

Open search quickly with Alt + Space.
Use AI Mode to get organized answers.
Upload images or files for analysis.
Select screen content with Google Lens.
Understand the current window or the whole screen through screen sharing.
Search local files, apps, and Google Drive content.

If you already rely on Google Search and want the experience to feel more like asking an assistant, Google app for desktop is worth a try.

Reference

Google app for desktop official page: https://search.google/google-app/desktop/

Sharing an Agent Skill for E-commerce Product Image Cutout and Standardization

Thu, 09 Apr 2026 21:43:50 +0800

product-cutout-normalize is an Agent Skill for product images.

It turns raw product photos into square transparent images with a consistent layout. The default rules are:

a 1024x1024 canvas
transparent background
preserve the full subject as much as possible
rotate vertical subjects to horizontal
center the subject
normalize visible subject width to 820px

It is a good fit for e-commerce assets, product libraries, and product detail page image preparation.

What this skill solves

Even after a basic cutout, product images often still have issues such as:

leftover white or light gray edges
inconsistent subject orientation
inconsistent canvas size
inconsistent subject scale
small specks in transparent regions

This skill handles them with a fixed workflow:

Use Gemini for cutout
Clean light-colored background from the borders
Remove small noise fragments
Rotate vertical images to horizontal
Scale the subject to a target width
Place it in the center of a transparent square canvas

That makes the output cleaner and more suitable for batch use.

When to use it

This is a good fit when you need to:

batch-process product photos
export transparent PNG assets
keep a consistent visual size
use a stable and repeatable workflow

If you only edit a few images or need to manually adjust each layout, this may not be the right tool.

Quick start

The most direct way to run it is:

`1`	`& ".\.venv\Scripts\python.exe" ".codex\skills\product-cutout-normalize\scripts\run_pipeline.py" "input_dir" "output_dir" --overwrite`

Before running it, make sure you have:

GEMINI_API_KEY
google-genai
Pillow

Install dependencies:

`1`	`.\.venv\Scripts\python.exe -m pip install google-genai pillow`

Set the environment variable:

`1`	`$env:GEMINI_API_KEY="your_api_key"`

Output rules

The default output is:

transparent-background PNG
1024x1024 canvas
820px subject width
centered subject
small noise fragments removed

So this is more than a simple background-removal script. It is closer to a product image cleanup script.

Main parameters

Common flags:

--model Default: gemini-2.5-flash-image
--canvas-size Output square canvas size, default 1024
--target-width Visible subject width, default 820
--min-component-pixels Transparent fragments smaller than this are removed, default 500
--overwrite Overwrite existing outputs

For example:

`1`	`& ".\.venv\Scripts\python.exe" ".codex\skills\product-cutout-normalize\scripts\run_pipeline.py" ".\input" ".\output" --canvas-size 1280 --target-width 960 --overwrite`

Workflow

The workflow is straightforward:

Use Gemini for cutout
Clean light-colored border background
Remove small fragments
Rotate vertical images to horizontal
Scale to the target width
Place the result on a transparent square canvas

How it differs from a basic cutout script

Compared with a basic background-removal script, it also handles:

subject orientation normalization
subject size normalization
canvas size normalization
small-fragment cleanup
outputs that are easier to place directly into an asset library

SKILL.md source

The full SKILL.md source is preserved below without modification:

---
name: product-cutout-normalize
description: Run a reusable Gemini product-image pipeline that removes backgrounds, preserves the full subject, rotates tall products to a horizontal orientation, centers them on a 1024x1024 transparent canvas, and normalizes the visible subject width to 820px. Use when the user wants a repeatable cutout-and-normalize workflow for product photos or asks to batch-process product images into standardized square PNG assets.
---

# Product Cutout Normalize

Use this skill when product photos need the same deterministic finishing pipeline:

- Gemini cutout from the original photo
- border cleanup to transparent
- preserve the full subject
- rotate to horizontal when the subject is taller than it is wide
- center on a `1024x1024` transparent canvas
- normalize the visible subject width to `820px`

## Quick Start

Run the bundled script:

```powershell
& ".\.venv\Scripts\python.exe" ".codex\skills\product-cutout-normalize\scripts\run_pipeline.py" "input_dir" "output_dir" --overwrite
```

Required environment:

- `GEMINI_API_KEY`
- `google-genai`
- `Pillow`

## Workflow

1. Confirm the request matches this standard pipeline. If the user asks for a different canvas size, subject width, or layout rule, pass explicit flags instead of changing the script.
2. Run the bundled script on the input directory.
3. If a result looks misaligned, inspect the alpha bounding box and small detached artifacts first; this pipeline already removes tiny alpha components by default.
4. Report the exact input and output directories used, plus any non-default flags.

## Script

Primary entry point:

- `scripts/run_pipeline.py`

Key flags:

- `--model`: Gemini image model, default `gemini-2.5-flash-image`
- `--canvas-size`: output square size, default `1024`
- `--target-width`: visible subject width after normalization, default `820`
- `--min-component-pixels`: remove detached alpha specks smaller than this, default `500`
- `--overwrite`: replace existing outputs in the destination directory

## Repo Integration

If the current project already has [`scripts/nano_banana_cutout.py`](/c:/Work/my_shop/scripts/nano_banana_cutout.py), prefer that repo script when the user wants the same pipeline inside this repository. Use the bundled skill script when the task is cross-project reuse or when you want the workflow to stay self-contained inside the skill.

scripts/run_pipeline.py source

The full scripts/run_pipeline.py source is preserved below without modification:

from __future__ import annotations

import argparse
import os
from collections import deque
from pathlib import Path

from PIL import Image

try:
    from google import genai
except ImportError as exc:  # pragma: no cover
    raise SystemExit(
        "Missing dependency: google-genai. Install it with "
        r"'.\.venv\Scripts\python.exe -m pip install google-genai'."
    ) from exc


PROMPT = (
    "Remove the entire background from this product photo and return only the product "
    "on a fully transparent background as a PNG. Keep the full product intact, preserve "
    "thin cable details, clean the inner loops and holes, and do not add any new objects "
    "or shadows."
)
DEFAULT_CANVAS_SIZE = 1024
DEFAULT_TARGET_WIDTH = 820
DEFAULT_MIN_COMPONENT_PIXELS = 500
SUPPORTED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".webp"}


def is_light_background_pixel(r: int, g: int, b: int) -> bool:
    brightness = (r + g + b) / 3
    spread = max(r, g, b) - min(r, g, b)
    return brightness >= 170 and spread <= 35


def to_pil_image(image_obj) -> Image.Image:
    if isinstance(image_obj, Image.Image):
        return image_obj
    pil_image = getattr(image_obj, "_pil_image", None)
    if isinstance(pil_image, Image.Image):
        return pil_image
    as_pil = getattr(image_obj, "pil_image", None)
    if isinstance(as_pil, Image.Image):
        return as_pil
    raise TypeError(f"Unsupported image object type: {type(image_obj)!r}")


def make_transparent_from_borders(image: Image.Image) -> Image.Image:
    rgba = image.convert("RGBA")
    width, height = rgba.size
    pixels = rgba.load()

    visited: set[tuple[int, int]] = set()
    queue: deque[tuple[int, int]] = deque()

    def push_if_bg(x: int, y: int) -> None:
        if (x, y) in visited:
            return
        r, g, b, _ = pixels[x, y]
        if is_light_background_pixel(r, g, b):
            visited.add((x, y))
            queue.append((x, y))

    for x in range(width):
        push_if_bg(x, 0)
        push_if_bg(x, height - 1)
    for y in range(height):
        push_if_bg(0, y)
        push_if_bg(width - 1, y)

    while queue:
        x, y = queue.popleft()
        for nx, ny in ((x - 1, y), (x + 1, y), (x, y - 1), (x, y + 1)):
            if 0 <= nx < width and 0 <= ny < height:
                push_if_bg(nx, ny)

    for x, y in visited:
        pixels[x, y] = (0, 0, 0, 0)

    return rgba


def remove_small_components(image: Image.Image, min_component_pixels: int) -> Image.Image:
    if min_component_pixels <= 0:
        return image

    rgba = image.convert("RGBA")
    alpha = rgba.getchannel("A")
    width, height = rgba.size
    alpha_pixels = alpha.load()
    rgba_pixels = rgba.load()
    visited: set[tuple[int, int]] = set()

    for y in range(height):
        for x in range(width):
            if alpha_pixels[x, y] == 0 or (x, y) in visited:
                continue

            queue: deque[tuple[int, int]] = deque([(x, y)])
            visited.add((x, y))
            component: list[tuple[int, int]] = []

            while queue:
                cx, cy = queue.popleft()
                component.append((cx, cy))
                for nx, ny in ((cx - 1, cy), (cx + 1, cy), (cx, cy - 1), (cx, cy + 1)):
                    if 0 <= nx < width and 0 <= ny < height:
                        if alpha_pixels[nx, ny] == 0 or (nx, ny) in visited:
                            continue
                        visited.add((nx, ny))
                        queue.append((nx, ny))

            if len(component) < min_component_pixels:
                for px, py in component:
                    r, g, b, _ = rgba_pixels[px, py]
                    rgba_pixels[px, py] = (r, g, b, 0)

    return rgba


def normalize_product_image(
    image: Image.Image,
    canvas_size: int,
    target_width: int,
) -> Image.Image:
    rgba = image.convert("RGBA")
    bbox = rgba.getchannel("A").getbbox()
    if bbox is None:
        return Image.new("RGBA", (canvas_size, canvas_size), (0, 0, 0, 0))

    subject = rgba.crop(bbox)
    if subject.height > subject.width:
        subject = subject.rotate(-90, expand=True, resample=Image.Resampling.BICUBIC)
        rotated_bbox = subject.getchannel("A").getbbox()
        if rotated_bbox is not None:
            subject = subject.crop(rotated_bbox)

    scale = target_width / subject.width
    subject = subject.resize(
        (target_width, max(1, int(round(subject.height * scale)))),
        Image.Resampling.LANCZOS,
    )

    canvas = Image.new("RGBA", (canvas_size, canvas_size), (0, 0, 0, 0))
    offset_x = (canvas_size - subject.width) // 2
    offset_y = (canvas_size - subject.height) // 2
    canvas.alpha_composite(subject, (offset_x, offset_y))
    return canvas


def finalize_product_image(
    image: Image.Image,
    canvas_size: int,
    target_width: int,
    min_component_pixels: int,
) -> Image.Image:
    transparent = make_transparent_from_borders(image)
    cleaned = remove_small_components(transparent, min_component_pixels)
    return normalize_product_image(cleaned, canvas_size=canvas_size, target_width=target_width)


def save_first_image_part(
    response,
    dst: Path,
    canvas_size: int,
    target_width: int,
    min_component_pixels: int,
) -> None:
    parts = getattr(response, "parts", None)
    if parts is None and getattr(response, "candidates", None):
        parts = response.candidates[0].content.parts

    if not parts:
        raise RuntimeError("Model returned no content parts.")

    for part in parts:
        inline_data = getattr(part, "inline_data", None)
        if inline_data is None and isinstance(part, dict):
            inline_data = part.get("inline_data")

        if inline_data is None:
            continue

        if hasattr(part, "as_image"):
            image = to_pil_image(part.as_image())
            dst.parent.mkdir(parents=True, exist_ok=True)
            finalize_product_image(
                image,
                canvas_size=canvas_size,
                target_width=target_width,
                min_component_pixels=min_component_pixels,
            ).save(dst)
            return

        data = getattr(inline_data, "data", None)
        if data:
            dst.parent.mkdir(parents=True, exist_ok=True)
            with open(dst, "wb") as handle:
                handle.write(data)
            with Image.open(dst) as image:
                processed = finalize_product_image(
                    image,
                    canvas_size=canvas_size,
                    target_width=target_width,
                    min_component_pixels=min_component_pixels,
                )
                processed.save(dst.with_suffix(".png"))
            if dst.suffix.lower() != ".png":
                dst.unlink(missing_ok=True)
            return

    raise RuntimeError("Model returned text only and no edited image.")


def process_image(
    src: Path,
    dst: Path,
    client,
    model: str,
    canvas_size: int,
    target_width: int,
    min_component_pixels: int,
) -> None:
    with Image.open(src).convert("RGBA") as image:
        response = client.models.generate_content(
            model=model,
            contents=[PROMPT, image],
        )

    save_first_image_part(
        response,
        dst,
        canvas_size=canvas_size,
        target_width=target_width,
        min_component_pixels=min_component_pixels,
    )


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Cut out product images with Gemini and normalize them to square transparent PNGs."
    )
    parser.add_argument("input_dir", type=Path)
    parser.add_argument("output_dir", type=Path)
    parser.add_argument("--model", default="gemini-2.5-flash-image")
    parser.add_argument("--canvas-size", type=int, default=DEFAULT_CANVAS_SIZE)
    parser.add_argument("--target-width", type=int, default=DEFAULT_TARGET_WIDTH)
    parser.add_argument("--min-component-pixels", type=int, default=DEFAULT_MIN_COMPONENT_PIXELS)
    parser.add_argument("--overwrite", action="store_true")
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    api_key = os.environ.get("GEMINI_API_KEY")
    if not api_key:
        raise SystemExit("Missing GEMINI_API_KEY environment variable.")
    if not args.input_dir.is_dir():
        raise SystemExit(f"Input directory does not exist: {args.input_dir}")
    if args.canvas_size <= 0:
        raise SystemExit("--canvas-size must be positive.")
    if args.target_width <= 0 or args.target_width > args.canvas_size:
        raise SystemExit("--target-width must be positive and no larger than --canvas-size.")
    if args.min_component_pixels < 0:
        raise SystemExit("--min-component-pixels must be >= 0.")

    args.output_dir.mkdir(parents=True, exist_ok=True)
    client = genai.Client(api_key=api_key)

    for src in sorted(args.input_dir.iterdir()):
        if not src.is_file() or src.suffix.lower() not in SUPPORTED_EXTENSIONS:
            continue
        dst = args.output_dir / f"{src.stem}.png"
        if dst.exists() and not args.overwrite:
            print(f"skip {dst}")
            continue
        process_image(
            src,
            dst,
            client,
            args.model,
            canvas_size=args.canvas_size,
            target_width=args.target_width,
            min_component_pixels=args.min_component_pixels,
        )
        print(dst)


if __name__ == "__main__":
    main()

Download attachment: product-cutout-normalize.7z

How to Use Google Nano Banana for Image Cutouts

Thu, 09 Apr 2026 20:10:48 +0800

This article uses a practical Python script to show how to call Google’s Nano Banana image editing capability for product-image cutouts.

The goal of this implementation is very clear:

Read product images from a directory
Call a Google image model to remove the background
Apply one more round of local transparent-background cleanup to the returned image
Export the final result as a transparent PNG

If you already have a batch of white-background product photos, headset images, or cable images and want to quickly generate transparent-background assets for e-commerce use, this approach is very direct.

What this code does

This script is mainly divided into 4 parts:

Define a prompt so the model understands it should remove the background, keep the subject intact, and avoid adding shadows
Call the google-genai image generation interface
Extract the image result from the model response
Use local logic to turn light-colored edge background areas transparent and reduce leftover halos

In other words, it does not simply send the image to the model and stop there. It combines model editing with local post-processing.

Before you run it

Install the dependencies first:

`1`	`.\.venv\Scripts\python.exe -m pip install google-genai pillow`

How to get GEMINI_API_KEY

GEMINI_API_KEY is the key used when calling the Gemini API. According to Google’s official quickstart, if you do not already have one, you can create it directly in Google AI Studio.

The process is straightforward:

Open Google AI Studio.
Sign in with your Google account.
Find the Get API key or API keys page.
Create a new API key.
Copy the generated key.
Configure it as a local environment variable for the script to read.

If there is no available project on the page yet, you typically need to finish project initialization first and then return to the API key page to create the key.

After you have the key, configure the environment variable:

`1`	`$env:GEMINI_API_KEY="your_api_key"`

If you are using cmd, you can write:

`1`	`set GEMINI_API_KEY=your_api_key`

If both GEMINI_API_KEY and GOOGLE_API_KEY are set, the runtime will usually prefer GOOGLE_API_KEY, so it is better to keep only one of them to avoid confusion.

Example directory structure

The script accepts two arguments:

input_dir: input image directory
output_dir: output image directory

For example:

images/
  product1.jpg
  product2.png

output/

How to run it

Assuming the script file is named cutout.py, run it like this:

`1`	`.\.venv\Scripts\python.exe .\cutout.py .\images .\output`

If you want to switch models, you can also pass it explicitly:

`1`	`.\.venv\Scripts\python.exe .\cutout.py .\images .\output --model gemini-2.5-flash-image`

The script will iterate over these file types in the input directory:

.jpg
.jpeg
.png
.webp

After processing, it will generate transparent-background PNG files with matching names in the output directory.

Core API call flow

The key code that actually calls Google Nano Banana is here:

response = client.models.generate_content(
    model=model,
    contents=[PROMPT, image],
)

Two pieces of content are passed in here:

A text prompt, PROMPT
A PIL.Image

The prompt asks the model to remove the full background from the product image, keep only the subject, and pay attention to a few important requirements:

Keep the full product intact
Preserve thin lines and cable details
Clean up inner holes and loop areas
Do not add new objects
Do not add shadows

Prompts like this have a big effect on cutout quality, especially for details such as earphone wires, transparent edges, and hollow regions.

Why local post-processing is still needed

After the model returns the result, the script does not save it directly. It also runs make_transparent_from_borders(image).

The idea behind this step is:

Start from the outer border of the image and find light-colored background pixels
Use breadth-first search to mark all connected light-colored regions
Convert those regions to transparent in one pass

The benefit is that it can further clean up leftover white edges, light gray backgrounds, and edge areas that are not clean enough.

The condition used to decide whether a pixel is background is here:

def is_light_background_pixel(r: int, g: int, b: int) -> bool:
    brightness = (r + g + b) / 3
    spread = max(r, g, b) - min(r, g, b)
    return brightness >= 170 and spread <= 35

In simple terms, this means:

The overall color must be bright enough
The RGB channel difference cannot be too large

This is especially suitable for product images with white, light gray, or near-solid-color backgrounds.

Full source code

The complete source code is preserved below so you can reuse or modify it directly:

from __future__ import annotations

import argparse
import os
from pathlib import Path
from collections import deque

from PIL import Image

try:
    from google import genai
except ImportError as exc:  # pragma: no cover
    raise SystemExit(
        "Missing dependency: google-genai. Install it with "
        r"'.\.venv\Scripts\python.exe -m pip install google-genai'."
    ) from exc


PROMPT = (
    "Remove the entire background from this product photo and return only the product "
    "on a fully transparent background as a PNG. Keep the full product intact, preserve "
    "thin cable details, clean the inner loops and holes, and do not add any new objects "
    "or shadows."
)


def is_light_background_pixel(r: int, g: int, b: int) -> bool:
    brightness = (r + g + b) / 3
    spread = max(r, g, b) - min(r, g, b)
    return brightness >= 170 and spread <= 35


def to_pil_image(image_obj) -> Image.Image:
    if isinstance(image_obj, Image.Image):
        return image_obj
    pil_image = getattr(image_obj, "_pil_image", None)
    if isinstance(pil_image, Image.Image):
        return pil_image
    as_pil = getattr(image_obj, "pil_image", None)
    if isinstance(as_pil, Image.Image):
        return as_pil
    raise TypeError(f"Unsupported image object type: {type(image_obj)!r}")


def make_transparent_from_borders(image: Image.Image) -> Image.Image:
    rgba = image.convert("RGBA")
    width, height = rgba.size
    pixels = rgba.load()

    visited: set[tuple[int, int]] = set()
    queue: deque[tuple[int, int]] = deque()

    def push_if_bg(x: int, y: int) -> None:
        if (x, y) in visited:
            return
        r, g, b, _ = pixels[x, y]
        if is_light_background_pixel(r, g, b):
            visited.add((x, y))
            queue.append((x, y))

    for x in range(width):
        push_if_bg(x, 0)
        push_if_bg(x, height - 1)
    for y in range(height):
        push_if_bg(0, y)
        push_if_bg(width - 1, y)

    while queue:
        x, y = queue.popleft()
        for nx, ny in ((x - 1, y), (x + 1, y), (x, y - 1), (x, y + 1)):
            if 0 <= nx < width and 0 <= ny < height:
                push_if_bg(nx, ny)

    for x, y in visited:
        pixels[x, y] = (0, 0, 0, 0)

    return rgba


def save_first_image_part(response, dst: Path) -> None:
    parts = getattr(response, "parts", None)
    if parts is None and getattr(response, "candidates", None):
        parts = response.candidates[0].content.parts

    if not parts:
        raise RuntimeError("Model returned no content parts.")

    for part in parts:
        inline_data = getattr(part, "inline_data", None)
        if inline_data is None and isinstance(part, dict):
            inline_data = part.get("inline_data")

        if inline_data is None:
            continue

        if hasattr(part, "as_image"):
            image = to_pil_image(part.as_image())
            dst.parent.mkdir(parents=True, exist_ok=True)
            make_transparent_from_borders(image).save(dst)
            return

        data = getattr(inline_data, "data", None)
        mime_type = getattr(inline_data, "mime_type", "")
        if data:
            dst.parent.mkdir(parents=True, exist_ok=True)
            with open(dst, "wb") as handle:
                handle.write(data)
            with Image.open(dst) as img:
                processed = make_transparent_from_borders(img)
                processed.save(dst.with_suffix(".png"))
            if dst.suffix.lower() != ".png":
                dst.unlink(missing_ok=True)
            return

    raise RuntimeError("Model returned text only and no edited image.")


def process_image(src: Path, dst: Path, client, model: str) -> None:
    with Image.open(src).convert("RGBA") as image:
        response = client.models.generate_content(
            model=model,
            contents=[PROMPT, image],
        )
    save_first_image_part(response, dst)


def main() -> None:
    parser = argparse.ArgumentParser(description="Use Nano Banana / Gemini image editing to cut out product images.")
    parser.add_argument("input_dir", type=Path)
    parser.add_argument("output_dir", type=Path)
    parser.add_argument("--model", default="gemini-2.5-flash-image")
    args = parser.parse_args()

    api_key = os.environ.get("GEMINI_API_KEY")
    if not api_key:
        raise SystemExit("Missing GEMINI_API_KEY environment variable.")

    client = genai.Client(api_key=api_key)
    exts = {".jpg", ".jpeg", ".png", ".webp"}

    for src in sorted(args.input_dir.iterdir()):
        if not src.is_file() or src.suffix.lower() not in exts:
            continue
        dst = args.output_dir / f"{src.stem}.png"
        process_image(src, dst, client, args.model)
        print(dst)


if __name__ == "__main__":
    main()

Good places to improve it further

If you plan to use this script for batch production, you can continue improving it in these directions:

Add retry logic so one failed image does not stop the whole batch
Add logs to make it easier to identify which image failed
Make background thresholds configurable
Support recursive scanning of subdirectories
Add a side-by-side preview of the original and processed result

Summary

If you only want the shortest explanation of how to use Google Nano Banana for cutouts, the core process is just three steps:

Install google-genai and Pillow
Set GEMINI_API_KEY
Pass the prompt and image to client.models.generate_content()

The value of this code is that it does more than just call the model. It also adds transparent-background post-processing, which makes it more suitable for direct product-image cutout work.