Veo 3.1 API Video Generation: First and Last Frames, References, 4K, and Polling

Veo 3.1 has entered the Gemini API, supports text, images and existing videos as input, and natively generates videos with audio. Instead of a normal interface that returns MP4 synchronously, it first creates a long-running operation, then polls the status and downloads the results. This article organizes the code according to this asynchronous model, while explaining the limitations between first and last frames, reference images, video extensions, resolution and duration.

Choose Veo 3.1, Fast or Lite first

Officially currently offering Veo 3.1, Veo 3.1 Fast and Veo 3.1 Lite routes. Standard version is suitable for picture consistency, complex shots and high-quality experiments. The Fast version is more suitable for interactive preview and batch screening of prompts. Lite is geared toward lower cost and faster production, but the feature set may not be the same as the highest resolution. The model may still be in preview, and the production system needs to lock the model ID and accept interface field changes. Don’t just write “Veo 3.1” in the console, the code needs to use the full model name currently listed in the official documentation.

Parameter combinations are more important than individual parameters

Veo 3.1 is available in 4, 6 or 8 seconds. Typically 8 seconds is required when using 1080p, 4K, reference or video extensions. Aspect ratio support 16:9 and 9:16. The video extension currently only outputs 720p. One video is generated per request. Frame rate is 24 fps. seed improves similarity but does not guarantee complete certainty. Verifying the parameter matrix before submitting can reduce the number of minutes to wait before receiving a failure.

Prepare API key and Python environment

Create a project in Google AI Studio with a Gemini API key. The key only contains environment variables:

1

$env:GEMINI_API_KEY = Read-Host "Gemini API key"

Create an independent virtual environment:

1
2
3
4

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
pip install google-genai pillow

Check package version:

1

python -m pip show google-genai

When the preview API changes, version information is more valuable for troubleshooting than “it still ran yesterday”.

First text generated video request

The following structure is based on the current interface of the official SDK. The model ID must be checked with the document before execution:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

import os
import time
from pathlib import Path

from google import genai
from google.genai import types

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

operation = client.models.generate_videos(
    model="veo-3.1-generate-preview",
    prompt=(
        "A quiet railway platform at dawn, light fog, "
        "slow dolly movement, realistic ambient sound, no text"
    ),
    config=types.GenerateVideosConfig(
        aspect_ratio="16:9",
        resolution="720p",
        duration_seconds=4,
    ),
)

while not operation.done:
    print("waiting", operation.name)
    time.sleep(10)
    operation = client.operations.get(operation)

video = operation.response.generated_videos[0].video
client.files.download(file=video)
video.save("veo-output.mp4")
print(Path("veo-output.mp4").resolve())

The SDK field may be updated with preview. If the attribute does not exist, first check the official example corresponding to the installed version.

Why the polling interval should not be too short

Generating a video takes longer than a text response. Polling every second will not make the video complete earlier, but will only increase the probability of API requests and throttling. The official REST example uses a 10 second interval, which is a reasonable starting point. Production tasks can use exponential backoff, but set the maximum interval and total timeout. Save the operation name to the database, and you can continue querying after the process is restarted. Don’t just leave the operation object in memory.

Save task state instead of blocking an HTTP request

After the web application receives the generation request, it first returns its own job ID. The background worker submits the Veo operation and saves the mapping:

1
2
3
4
5
6
7

{
  "job_id": "video_20260728_001",
  "operation_name": "operations/example",
  "status": "running",
  "model": "veo-3.1-generate-preview",
  "created_at": "2026-07-28T12:00:00Z"
}

The front end queries its own job endpoint without directly exposing the Gemini key or complete operation data. After the worker crashes, read operation_name and continue polling.

Image generation video uses the first frame

The first frame determines the composition of the video. Choose images with clear subjects, intact edges, and no watermarks or unnecessary text.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

from PIL import Image

first_frame = Image.open("first-frame.png")

operation = client.models.generate_videos(
    model="veo-3.1-generate-preview",
    prompt="The camera slowly moves closer while leaves sway in the wind",
    image=first_frame,
    config=types.GenerateVideosConfig(
        aspect_ratio="16:9",
        resolution="720p",
        duration_seconds=8,
    ),
)

When the image aspect ratio is too different from the output, the model needs to crop or fill in the edges. Processing the canvas locally to 16:9 or 9:16 first will usually result in more controllable results.

The first frame plus the last frame is used for interpolation transition

Veo 3.1 supports specifying start and end screens. The last frame is not an independent reference picture, but the picture to which the video will eventually transition.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

first_frame = Image.open("start.png")
last_frame = Image.open("end.png")

operation = client.models.generate_videos(
    model="veo-3.1-generate-preview",
    prompt="A smooth cinematic transition from morning to night",
    image=first_frame,
    config=types.GenerateVideosConfig(
        last_frame=last_frame,
        aspect_ratio="16:9",
        resolution="1080p",
        duration_seconds=8,
    ),
)

When the difference in the positions of the subjects in the two images is too large, the model may deform to complete the transition. Aligning the camera angle, subject scale, and horizon line first is more effective than adding cue words.

Reference image used to maintain character or product

Veo 3.1 can use up to three asset reference images. Reference images are great for giving different perspectives on the same character, product, or style. If the clothing, colors and proportions of the three pictures conflict with each other, it will reduce the consistency. When using product images, keep the complete outline and avoid using the brand text as a background texture. Reference images, 1080p or 4K requests are subject to the official limit of 8 seconds. Whether the Lite model supports the corresponding reference image type shall be subject to the current parameter table.

4K does not mean changing the 720p parameter into a string

4K requires a standard Veo 3.1 route with a duration requirement of 8 seconds. It increases build time, download size, and post-processing resources. Verify footage in Fast or 720p first, then generate 4K on passing cue words.

1
2
3
4
5

config = types.GenerateVideosConfig(
    aspect_ratio="16:9",
    resolution="4k",
    duration_seconds=8,
)

Vertical 4K availability on current models and regions should be subject to request return and official form. Do not verify the parameter combination for the first time before batch submission.

Let the prompts describe both the picture and the sound

Veo 3.1 natively generates audio, and prompts can describe ambient sounds, dialogue, and musical mood. Prompts include at least subject, action, scene, lens, light and sound.

1
2
3
4
5

A close-up of a ceramic cup on a wooden table.
Steam rises slowly while rain hits the window behind it.
The camera performs a gentle clockwise orbit.
Warm indoor light, shallow depth of field.
Audio: soft rain, distant thunder, no speech, no music.

“Cinematic” is not a substitute for camera movement and compositional instructions. When no text is needed, clearly write no text, but still have to check the finished film.

The video extension can only use Veo generated videos

The extended input is the Video object in the previously generated result, not an arbitrary MP4 upload. The extension uses the last second, or approximately 24 frames, to continue the action. When there is no sound in the last second of the original video, the voice usually cannot continue naturally. Put the actions and sounds you want to continue at the end, and then ask for the next paragraph. Extended output currently only supports 720p. Multiple expansions will accumulate visual and audio drift, and feature films should manage shots in editing software.

The core of the REST request is operation name

When not using SDK, you can call predictLongRunning.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

curl -s \
  "https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-generate-preview:predictLongRunning" \
  -H "x-goog-api-key: ${GEMINI_API_KEY}" \
  -H "Content-Type: application/json" \
  -X POST \
  -d '{
    "instances": [{
      "prompt": "A calm ocean at sunrise, slow aerial shot, natural waves"
    }],
    "parameters": {
      "aspectRatio": "16:9",
      "resolution": "720p",
      "durationSeconds": 4,
      "sampleCount": 1
    }
  }'

name in the response is used for subsequent queries. Don’t put API keys into URL query parameters or front-end JavaScript.

Polling REST operation

1
2
3

curl -s \
  -H "x-goog-api-key: ${GEMINI_API_KEY}" \
  "https://generativelanguage.googleapis.com/v1beta/${OPERATION_NAME}"

Continue waiting when done is false. After done is true, check whether it is response or error. Don’t assume completion is a guaranteed success. Save the complete error structure and request ID to avoid showing just “Build failed”.

Download URL still requires API key

The completion response gives the generated video URI. Use -L to follow the redirect:

1
2
3
4

curl -L \
  -H "x-goog-api-key: ${GEMINI_API_KEY}" \
  -o veo-result.mp4 \
  "${VIDEO_URI}"

Check the file type and duration after downloading:

1
2
3

ffprobe -v error \
  -show_entries format=duration,size \
  -of json veo-result.mp4

Don’t just use the file extension to determine whether the download was successful, the incorrect JSON may also be saved as .mp4.

Set idempotent key

for build task After the client times out, do not resubmit the same prompt word immediately. First check whether your job table already has an operation. Generate idempotent keys with user ID, material hash, parameters and business request ID. Only one upstream operation is allowed to be created for the same key. This prevents one click from generating two billable videos.

Cost control starts with two-stage generation

The first stage used Fast, 720p, and 4 seconds to screen prompts. The second stage only upscales selected compositions to 8 seconds, 1080p or 4K. Record the model, resolution, duration, operation and final file size. Calculate the cost based on successful production, not just the single API unit price. Content moderation failures, parameter failures, and user cancellations are also factored into the budget.

Common parameter errors

4K with 4 seconds would violate the duration requirement. Extended video requests for 1080p violate the extended resolution limit. Reference images exceeding three will be rejected. The first and last frames only provide the last frame and interpolation cannot be formed without the first frame. If the preview model ID expires, it will be returned that the model does not exist. Character generation limits may also vary by region and input mode.

401, 403 and 429 are processed separately

401 first checks whether the key has entered the current process. 403 Checks project permissions, regions, model access, and organizational policies. 429 Checks requests per minute, concurrency, and project quotas. Do not retry on 403 infinite backoff. Quota errors should be queued or the user should be prompted to try again later.

Do a media quality check after downloading

Check container format, duration, resolution, frame rate and audio track.

1
2
3
4

ffprobe -v error \
  -show_streams \
  -show_format \
  -of json veo-result.mp4 > veo-result.json

Then extract the first frame, middle frame and last frame:

1
2
3

ffmpeg -i veo-result.mp4 \
  -vf "select='eq(n,0)+eq(n,96)+eq(n,191)'" \
  -vsync 0 frame-%02d.png

The frame number should be adjusted according to the real duration and fps.

Security and content rights cannot be left to parameters instead

Confirm usage rights before uploading images of people, brands, and products. Do not generate material intended to deceive, harass, or impersonate. Original images, prompts, and generated videos may contain personal information. Set the retention period for temporary files, and do not store images in base64 in logs. User-facing products also require content moderation, reporting, and removal paths. All generated images and videos will be subject to platform watermarks and policies.

Task acceptance before going online

• operation name can be persisted.
• The worker can continue polling after restarting.
• The download follows the redirect and carries the authentication.
• 720p, 1080p, and 4K parameter combinations were tested separately.
• The number of first and last frames and reference pictures meets the limit.
• Repeat requests will not be billed again.
• ffprobe can confirm audio and video streams.
• The failed task retains the error structure without saving the key.
• Users can delete materials and generated results.

The engineering difficulties of Veo 3.1 API are mainly in asynchronous tasks, parameter matrix and result management. First run through the 4-second 720p process, and then add the first and last frames, reference images, and 4K. It will be easier to locate the problem than submitting all advanced parameters at once.

Official information

• Google: Generating videos using Veo 3.1
• Google: Veo 3.1 model description