HeyGen Open-Sources HyperFrames Framework with Keyframes for AI Video Agents

HeyGen has open-sourced HyperFrames, an HTML-to-video framework that enables AI agents to generate videos using skills and a visual Keyframes editor for timeline animation control with code synchronization.

This setup supports templates for product launches, site tours, and motion graphics while allowing manual visual edits to update AI-readable GSAP code directly.

Overview of HyperFrames Framework

HyperFrames is an open-source framework under the Apache 2.0 license that lets users and AI agents create deterministic MP4 videos by writing HTML, CSS, and JS with GSAP for animations. The project originated at HeyGen but is maintained for community use through its GitHub repository. Compositions rely on plain HTML with data attributes and registered GSAP timelines exposed via window.__timelines. This structure produces consistent output on every render because all parameters are explicitly defined in code.

The framework targets developer and agent-driven workflows rather than general consumer editing. It keeps the source material fully inspectable and modifiable by both manual and automated processes. Standard web technologies form the base, which reduces barriers for teams already familiar with front-end code. The deterministic output supports repeated generation without unexpected variations.

When choosing HyperFrames, evaluate whether the project requires reproducible video output and direct code access for AI agents. The framework suits environments where teams maintain version control over animation logic and need to apply updates across multiple video instances without re-authoring assets. It proves less suitable for one-off creative projects that prioritize rapid visual experimentation over code precision.

In a conditional scenario, a development group might define a product video layout using standard HTML elements and GSAP calls, then render variations by modifying numeric values in the source files. This method allows the same base composition to serve different campaign requirements through targeted code edits rather than separate design files.

A primary limitation stems from the explicit requirement to define all animation parameters in code, which extends setup time for intricate sequences compared to timeline-based commercial editors. Typical errors include omitting the registration of GSAP timelines in the window object, resulting in silent failures during rendering where expected motion does not appear. Verification of the timelines object before each render prevents this issue.

Another constraint involves dependency management, as the framework relies on specific versions of GSAP and related libraries that must remain compatible during updates. Users should document their exact dependency tree to avoid breaks when the underlying packages evolve. The Apache 2.0 license grants broad usage rights but requires compliance with attribution rules for any redistributed modifications.

Teams benefit from the framework when their workflow already incorporates code reviews and automated testing, as the HTML structure integrates naturally with existing development pipelines. This alignment reduces context switching between design and engineering tasks.

AI Agent Integration and Skills

HyperFrames supplies installable skills that equip AI agents with established patterns for video creation. Agents install them via npx skills add heygen-com/hyperframes and then invoke specific slash commands during prompting. The /product-launch-video workflow accepts a website URL, brief, or script and produces a launch or promo video or a site tour that may include voiceover or an avatar. The /motion-graphics skill generates short design-led clips that follow motion design conventions.

Agents such as Claude, Codex, or Hermes receive context from these skills to follow template structures without authoring every HTML element from scratch. The skills reduce the need for agents to invent workflows for recurring video formats. Installation occurs on demand, so only relevant patterns load into the agent session at any given time.

Selection of skills depends on the video type and the agent's prior exposure to similar tasks. Use the provided slash commands when the goal matches one of the documented templates, such as product launches or site reviews. Custom prompting without skills often leads to inconsistent structures that require additional correction steps.

In a conditional case, an agent prompted with a site URL and the /product-launch-video command might generate an initial composition featuring animated UI elements and a voiceover script derived from the page content. The output then serves as a starting point for further refinement through either visual tools or additional agent instructions.

Limitations include the need for agents to reference the exact skill documentation, as deviations from the expected command format can produce incomplete or malformed code. Typical errors involve attempting to use skills without prior installation, which causes the agent to fall back to generic responses lacking the template structure. Always confirm skill availability in the current session before issuing commands.

The skills function as modular extensions rather than permanent modifications to the agent, allowing selective activation based on project needs. This modularity supports experimentation with different video styles without committing to a single approach.

Documentation for prompting patterns appears in the official guide, which outlines expected inputs and outputs for each workflow. Following these patterns improves the quality of initial generations and reduces iteration cycles.

Keyframes Tool and Timeline Editing

The Keyframes tool presents a visual Studio with a timeline that shows diamond markers for GSAP keyframes synthesized from .to(), .from(), and .fromTo() calls. Users select elements and adjust properties such as Move X/Y, Scale, Opacity, timing, and easing in the Design Panel. All visual changes update the generated GSAP code immediately, and the updated code appears in the Code tab for review. This synchronization keeps manual edits compatible with subsequent AI agent processing.

The editor allows direct control over object trajectories and motion smoothness on the timeline. Changes remain literal GSAP tweens that agents can read, debug, or regenerate. The Design Panel exposes only properties that map directly to GSAP tween parameters, ensuring edits translate without loss of information.

Adopt the Keyframes editor when precise timing adjustments or trajectory refinements are required beyond what an initial agent prompt can achieve. It serves projects where visual feedback accelerates iteration compared to pure code editing. The tool fits less well when the animation logic involves heavy computation or conditional branching that cannot be expressed as simple tweens.

In a conditional workflow, a user might open a composition containing multiple .to() calls, select an element on the timeline, and increase its scale value while shortening the duration. The Code tab then reflects the revised parameters, which an agent can subsequently review for consistency with the overall video brief.

Limitations center on the requirement that tweens remain literal rather than derived from helper functions or loops. Helper or computed tweens may require unrolling for direct edits. Typical errors include attempting to edit non-tween elements or overlooking easing functions, which can produce abrupt motion in the final render. Always verify that selected properties correspond to active GSAP calls before saving changes.

The immediate code reflection supports a hybrid process where visual adjustments feed back into agent-driven refinements. This loop maintains the code as the authoritative source while providing intuitive controls for motion details.

Users should test the editor on simple compositions first to understand how property changes propagate through the timeline markers. This practice reveals the exact mapping between visual controls and code output.

Advanced Animation Features (Arc Motion and Gestures)

Arc Motion converts straight x/y GSAP tweens into curved paths using the MotionPathPlugin. Users adjust curviness from zero for straight lines to higher values for more pronounced arcs and can enable auto-rotate to align elements along the path. The editor writes the corresponding motionPath configuration into the code so the curved motion persists after export. This produces animations that appear more natural in the final rendered video.

Gesture recording captures additional movements and converts them into code sequences that agents can incorporate. The feature extends the range of expressiveness available within the same code-based workflow. Both Arc Motion and gesture tools update the underlying GSAP configuration without requiring manual code entry.

Choose these features when straight-line motion appears mechanical or when the video requires organic-feeling trajectories. They apply best to elements that benefit from physical simulation, such as floating objects or character movements. Straightforward linear animations do not require these additions and may suffer from unnecessary complexity if forced into curved paths.

In a conditional example, an initial linear tween for an icon might receive Arc Motion with a curviness value of 0.5 and auto-rotate enabled. The resulting code includes the motionPath object, which the agent can later adjust if the curve needs modification for the specific scene.

Limitations arise because Arc Motion applies only to x/y position tweens and does not affect other properties such as rotation or opacity directly. Typical errors include setting extreme curviness values that cause elements to exit the frame unexpectedly or neglecting to test the rendered output after gesture recording. Review the generated motionPath configuration in the Code tab to confirm the intended path before final rendering.

The translation between visual adjustments and code remains consistent, allowing agents to extend or correct advanced motions without format conversion. This consistency supports iterative development across both manual and automated steps.

Documentation on the MotionPathPlugin parameters helps users predict how visual settings will appear in the final video. Testing small adjustments incrementally avoids large-scale revisions later in the process.

Figma Import and MCP

The /figma skill imports assets, tokens, components, and storyboard sections from Figma through REST or CLI interfaces. Imported elements are reconstructed as motion frames or states ready for animation within HyperFrames. MCP operates as a hosted Model Context Protocol server that supports motion animations and shaders. Access requires a HeyGen account for authentication and credits, while local open-source use does not.

Imported Figma content can be further adjusted in the Keyframes editor or passed to an agent for animation code generation. The skill preserves design tokens and component structure during transfer. This capability reduces duplication of design work when moving from static layouts to animated compositions.

Apply the Figma import when existing design files contain the visual foundation for the video. The skill works effectively for projects that begin with component libraries or storyboard outlines already defined in Figma. It offers limited value when designs exist only as loose references or require substantial recreation before import.

In a conditional workflow, a designer might export a set of UI components and a storyboard from Figma, then invoke the /figma skill to bring those elements into a HyperFrames composition. The resulting frames can receive motion assignments through either the Keyframes tool or an agent prompt.

Limitations include the hosted nature of MCP, which ties advanced features to an active HeyGen account and associated credits. Local use of the open-source framework bypasses this requirement but excludes MCP-specific capabilities. Typical errors involve mismatched component structures during import, which can produce incomplete motion frames that require manual correction in the editor. Validate the imported elements against the original Figma file before proceeding with animation.

The distinction between local and hosted paths affects only the MCP features, leaving core import and editing functions available in both modes. Teams can therefore select the path that matches their account status and collaboration needs.

Review the skill documentation for supported Figma element types to ensure compatibility before starting the import process. This step prevents partial transfers that complicate subsequent animation work.

Workflow for AI Agents: From Prompt to Render

An agent begins by applying a skill to a prompt that specifies the video type and input details. The skill generates the initial HTML structure and GSAP timelines according to the chosen template. The resulting composition opens in the Keyframes editor for visual review and adjustments to timing, paths, or properties. Every edit produces updated code that the agent can access directly.

The agent can then be prompted to correct visual issues in the code or to add new elements if the initial output requires expansion. This creates a closed loop between visual editing and code-based iteration. The code remains the single source of truth throughout, so changes made visually stay available for agent review without format conversion.

Follow this workflow when the project involves both initial generation from a brief and targeted refinements for animation quality. It supports teams that combine agent assistance with occasional manual oversight. Purely manual or purely agent-driven approaches may suffice for simpler tasks but lack the hybrid efficiency for complex videos.

In a conditional sequence, an agent first receives a prompt with the /motion-graphics skill and a design description. After generating the base code, the user opens the composition in Keyframes to refine arc paths, then returns the updated code to the agent for final validation and rendering instructions.

Limitations include the dependence on straightforward tween structures for seamless visual editing. Complex conditional logic may require the agent to handle refinements entirely in code. Typical errors involve skipping the visual review step, which can leave timing issues undetected until the final render. Incorporate a review checkpoint after each major edit to maintain quality.

The process scales to multiple iterations because each step preserves the code representation. Agents can reference previous versions through the same file structure, enabling systematic improvement without losing prior work.

Establish clear handoff points between agent generation and visual editing to avoid overlapping changes that create conflicts. Documenting these points helps maintain consistency across team members or repeated sessions.

Usage Options: Local vs. Hosted

Local usage relies on the open-source CLI and renderer to handle composition creation and MP4 output on the user's machine. This route requires no external accounts and keeps all processing under direct control. The hosted MCP path adds integrated AI interactions and extra animation capabilities but connects to HeyGen services and needs an account. The framework itself functions separately from HeyGen's main video editor.

Teams prioritizing full autonomy select the local installation, while those needing seamless agent assistance may prefer the hosted option. Documentation as of May 8, 2026 covers both approaches with the same core features. The choice depends on whether the workflow benefits from hosted AI context or requires offline operation.

Evaluate local versus hosted based on data sensitivity, collaboration requirements, and available credits. Local use supports environments with strict access controls, whereas hosted MCP enables direct prompting within the same interface as other HeyGen tools. Neither option replaces the need for initial skill installation and template understanding.

In a conditional setup, a solo developer might install the CLI locally to test a motion graphics composition without account creation, then switch to MCP if agent-assisted refinements become necessary for a client project.

Limitations of the local path include the absence of MCP-hosted features such as advanced shaders. The hosted path introduces dependency on account status and potential credit consumption. Typical errors include assuming all features work identically in both modes, which leads to missing functionality during local testing. Confirm the required capabilities before committing to one installation method.

Both paths share the same skill system and Keyframes editor, ensuring consistent core behavior. Users can prototype locally and migrate to hosted when additional AI integration proves beneficial.

Begin with a local installation and a test prompt for a simple motion graphics video to assess baseline performance. This initial evaluation informs whether to incorporate the MCP server based on observed workflow requirements.