claude-video: Let Claude Watch Videos with /watch, Extract Frames, Transcribe, and Answer Questions

Wed, 08 Jul 2026 08:15:05 +0800

Claude can read webpages, code, and documents, but it does not watch video by default. If you paste a YouTube link, it may rely on the title, page text, or captions. That is not enough when the important content is on screen: a product demo, a screen recording, a bug reproduction, an ad hook, or a code error.

bradautomates/claude-video adds a /watch capability. You provide a video URL or a local file path, ask a question, and the tool downloads only what it needs, extracts captions, samples frames, and hands the material to Claude.

What It Is Good For

This is not just a video downloader or transcript summarizer. Its value is giving the AI both the visual frames and the spoken text.

Useful cases include:

analyzing YouTube, TikTok, Loom, Vimeo, X, Instagram, and other public videos;
summarizing long videos or courses;
turning tutorials into notes;
checking what was actually shown in a launch, ad, or competitor video;
diagnosing bug reproduction recordings;
asking what happened at a specific timestamp;
finding UI issues, error messages, or steps in a screen recording.

Example:

`1`	`/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?`

Local recording:

`1`	`/watch ~/Movies/screen-recording.mp4 when does the UI break?`

How It Works

1. Video plus question

The input can be a URL or a local file. Because it uses yt-dlp, it supports many sites, including YouTube, Loom, TikTok, X, Instagram, and many others. Local formats include .mp4, .mov, .mkv, and .webm.

2. Captions first

The tool checks for native or auto-generated captions before downloading video. In transcript mode, a captioned URL can return without downloading the video, which is faster and cheaper.

3. Frame extraction with ffmpeg

When the question needs visuals, it uses ffmpeg to extract frames. Detail modes control the tradeoff:

efficient: fast keyframe extraction;
balanced: default, scene-aware frames;
token-burner: more complete scene coverage;
transcript: transcript only, no frames.

Frames are 512px wide by default and sized for Claude Read compatibility. Increase resolution when Claude needs to read small text, slides, terminals, or code.

4. Whisper fallback

If a video has no captions, claude-video can extract audio and transcribe it with Whisper. It prefers Groq whisper-large-v3 and also supports OpenAI whisper-1.

Most public videos can be handled through captions for free. Whisper keys are only needed when there is no usable caption track.

5. Frames and transcript go to Claude

The tool outputs timestamped transcript text and image frame paths. Claude reads the frames and answers from what is actually visible and spoken, not just from a title or summary.

Why Frame Budget Matters

Video analysis is expensive because every frame is an image. A 30-second clip can be scanned densely, but a 30-minute video can burn context quickly.

Default frame budgets are roughly:

up to 30 seconds: about 30 frames;
30 seconds to 1 minute: about 40 frames;
1 to 3 minutes: about 60 frames;
3 to 10 minutes: about 80 frames;
over 10 minutes: about 100 frames in capped modes, with a sparse-scan warning.

If you care about a specific segment, pass a time window:

`1`	`/watch https://youtu.be/abc --start 2:15 --end 2:45`

This is usually more accurate and cheaper than scanning a whole long video.

Deduplication Helps

Slides, screen recordings, and product demos often hold the same frame for a long time. Sending near-identical frames wastes image tokens.

claude-video deduplicates frames by default. It uses ffmpeg to scale frames to tiny grayscale thumbnails, compares brightness differences, and drops near-duplicates so the frame budget goes to distinct content.

Install

For Claude Code:

1
2

/plugin marketplace add bradautomates/claude-video
/plugin install watch@claude-video

For Codex, Cursor, Copilot, Gemini CLI, and other Agent Skills hosts:

`1`	`npx skills add bradautomates/claude-video -g`

-g installs globally for the current user. Without it, the skill is scoped to the current project.

First Run

On the first /watch, the skill checks for ffmpeg, yt-dlp, and optional Whisper configuration.

macOS can use brew install ffmpeg yt-dlp;
Linux prints apt, dnf, or pipx commands;
Windows prints winget or pip commands;
Whisper fallback uses GROQ_API_KEY or OPENAI_API_KEY in ~/.config/watch/.env.

If the video has captions, you usually do not need a Whisper key.

Practical Tips

Start with transcript or efficient for long videos. Use --start and --end when you know the relevant moment. Increase resolution only when on-screen text matters. Clean up local working files for private videos. And respect platform access rules; this is for analysis, not bypassing paywalls or copyright limits.

Who Should Use It

It fits people who watch many technical videos, courses, product launches, ads, competitor demos, or bug recordings. It is less useful if you only want to download videos, do not want command-line tools, need professional frame-perfect video analysis, or cannot send the video content to an AI tool.

Conclusion

claude-video turns video into materials Claude can understand: captions for audio, frames for visuals, and timestamps for grounding. If video is part of your information workflow, /watch can save a lot of manual scrubbing.

Sources:

Video Analysis on KnightLi Blog