subh05sus avatar

video-understanding-skill

0 subscribers
PythonSwift

Agentic video understanding for Claude: a Think-Act-Observe loop over frames, speech, sound and on-screen text

Created Oct 2026

MIT license

Live activities

The project README has been overhauled to be more approachable for new users, moving from dense technical details to a clear, quick-start guide organized by effort level. Advanced technical documentation and implementation details have been moved into a collapsible section, improving overall readability without sacrificing depth. Improved documentation for everyone

Added new effort level controls (quick, normal, deep, max) to the vu teardown, overview, and budget tools. Users can now balance analysis depth with resource usage, with deep offering part-level breakdowns and max providing frame-by-frame granularity. This change provides greater flexibility in how thoroughly the system processes video content. Adjusting settings

We've added a comprehensive deep teardown pipeline to the video understanding skill that decomposes videos into detailed technical components, including native editing cuts, motion tracking via GSAP eases, typographic synchronization, and audio analysis. This system runs six parallel analysis lenses, leveraging updated Swift-based vision tools and a dedicated NumPy/OpenCV/Pillow environment, to provide a structured digest and style guide for AI processing. This granular data extraction allows for much deeper programmatic insights into how complex videos are structured and produced. What if I told you that... We can now deconstruct every frame of a video automatically.

Updated the vu video understanding tool to share its voice-detection model (ggml-silero-v5.1.2.bin) with reelcut's watch tool. By checking for the model in reelcut's existing directory structure before attempting a download, we reduce redundant storage usage and improve integration between these related tools. Shared resources

Added a new 'vu.py' toolset that enables Claude to interact with video files using a Think-Act-Observe loop. By combining native Python processing with ffmpeg, whisper.cpp, and macOS Vision, the agent can now perform tasks like searching transcripts, OCR, frame-grid analysis, and motion mapping. This update includes a comprehensive skill guide and predefined recipes for tasks ranging from reel critiques to complex video-to-step breakdowns. Agentic video understanding

- End of feed -