Firefly Video Model

Firefly Video Model

Adobe positions Firefly Video as the first commercially-safe AI video generation API. It generates 1080p video clips from text prompts or images. Output is designed to be commercially safe — trained on licensed and public-domain content, the same trust posture as Firefly's image models.

This is a newer endpoint than image generation; rate limits are lower, generation takes longer, and prompt patterns differ. This skill encodes patterns from early production deployments.

When to Use This Skill

Use this skill when:

• A campaign needs short motion clips that have to be commercially safe
• Static hero images need motion variations for social / pre-roll
• A storyboard concept needs an animatic
• The user mentions "video", "motion graphics", "animation", "Firefly Video"

Do NOT use this skill when:

• The user wants long-form video editing — Firefly Video generates clips (typically 5-10s), not full edits
• IP safety isn't a requirement and another video generation model is acceptable to the customer
• The user wants live-action footage of real people — Firefly Video generates synthetic content

What Firefly Video Can and Can't Do

Can

• Generate 1080p clips from text prompts (up to ~10s typical)
• Generate 1080p clips from an image as motion seed (image-to-video)
• Maintain temporal coherence over short durations
• Match camera moves (pan, zoom, dolly) via prompt
• Output in standard aspect ratios (16:9, 9:16, 1:1)

Can't (today)

• Generate clips longer than ~10s in a single call
• Generate complex multi-shot sequences
• Lip-sync to provided audio
• Generate clips with recognizable real people
• Match a specific brand custom-trained style (no video custom models as of this writing)

If the use case needs long-form, multi-shot, or character lip-sync, plan to use Firefly Video for B-roll / inserts only, with the rest of the edit assembled in Premiere or After Effects.

Step 1 — Submit the Video Generation Job

curl --silent -X POST 'https://firefly-api.adobe.io/v3/videos/generate' \
  -H "Authorization: Bearer $FIREFLY_SERVICES_ACCESS_TOKEN" \
  -H "X-Api-Key: $FIREFLY_SERVICES_CLIENT_ID" \
  -H 'x-model-version: video1_standard' \
  -H 'Content-Type: application/json' \
  -d '{
    "prompt": "a slow cinematic dolly shot through a sunlit forest, dappled light, gentle camera motion",
    "sizes": [{"width": 1920, "height": 1080}]
  }'

The x-model-version: video1_standard header selects the video model. sizes is an array of {width, height} objects.

Returns the async job pattern (jobId, statusUrl, cancelUrl). Poll the status URL with Authorization: Bearer <token> and x-api-key: <client_id> — note that video jobs take 1-3 minutes typically (occasionally longer under load), not the ~10 seconds image jobs take. Use a longer polling interval (5-15s) and a generous max timeout (10-15 minutes).

Step 2 — Request Shape

{
  "prompt": "string — describe motion + scene + style + camera",
  "negativePrompt": "string — characteristics to steer away from",
  "sizes": [{"width": 1920, "height": 1080}],
  "seeds": [12345],
  "bitRateFactor": 18,
  "image": {
    "conditions": [
      {"source": {"uploadId": "abc-123"}, "placement": {"position": 0}}
    ]
  },
  "videoSettings": {
    "cameraMotion": "camera zoom in",
    "shotAngle": "low angle shot",
    "shotSize": "close-up shot",
    "promptStyle": "cinematic"
  }
}

Send the x-model-version: video1_standard header with the request to select the video model.

videoSettings enum values

Structured videoSettings.cameraMotion is the deterministic way to control the camera — prefer it over prompt-text camera language when the move maps to one of the enum values.

Step 3 — Prompt Patterns for Video

Video prompts differ from image prompts. The model needs to know motion, not just scene composition.

Structure: subject + setting + motion + camera + style

[Subject] in [setting], [what the subject is doing], [camera motion], [visual style]

Examples

Camera move vocabulary the model understands

For pans, zooms, tilts, locked-down, and handheld moves, the typed videoSettings.cameraMotion enum (Step 2) is the deterministic option. Use prompt-text camera language for moves outside the enum:

• dolly in / dolly out — camera moves toward / away
• pan left / pan right — camera rotates horizontally
• tilt up / tilt down — camera rotates vertically
• tracking shot — camera follows subject
• static shot — camera stays still
• aerial shot / drone shot — top-down or overhead
• whip pan / crash zoom — fast, aggressive moves

Anti-patterns

Avoid:

• "Suddenly the subject does X" — Firefly Video struggles with discrete narrative events; a single sustained motion works better
• Multiple simultaneous motions ("the dog runs while the camera spins while the background changes") — pick one dominant motion
• Long sequences ("first this happens, then this happens") — generate two clips instead

Step 4 — Image-to-Video

Provide a source image and Firefly will generate motion from that frame:

{
  "prompt": "gentle camera pull-back revealing more of the scene, subtle wind moving the trees",
  "image": {
    "conditions": [
      {"source": {"uploadId": "$SOURCE_IMAGE_ID"}, "placement": {"position": 0}}
    ]
  },
  "sizes": [{"width": 1920, "height": 1080}]
}

(Send with the x-model-version: video1_standard header.)

Each keyframe condition requires both source and placement. "position": 0 anchors the image as the first frame; "position": 1 anchors it as the last frame. Supplying two conditions — one at position 0 and one at position 1 — generates a clip that starts on one approved still and ends on another.

Image-to-video is the production pattern for motion variants of approved hero stills — you've already approved the static image; the video is just adding motion to it.

The source image must match the target aspect ratio. Generate or expand the image first if needed.

Step 5 — Output and Download

Successful job response:

{
  "status": "succeeded",
  "result": {
    "size": {"width": 1920, "height": 1080},
    "outputs": [
      {
        "seed": 646214641,
        "video": {
          "url": "https://pre-signed-cdn-url..."
        }
      }
    ]
  }
}

Each output carries exactly seed and video.url; the result also carries the size. Duration and container metadata are not part of the response — probe the downloaded file (e.g. ffprobe) if your pipeline needs them.

Download the MP4 immediately and re-host. URLs expire (typically 1 hour).

Video files are larger than images — a 1080p 5-second clip is typically 5-15MB. Plan storage and CDN bandwidth accordingly.

Production Patterns

Pattern: Motion variants of approved heroes

1. Image team approves a hero still
2. Image-to-video, 3 different motion prompts (subtle, medium, dramatic)
3. Creative team picks one
4. Picked variant goes to edit pipeline as B-roll

This pattern avoids the "is the moving version actually approvable" risk — the still is locked first, then motion is layered.

Pattern: Animatic generation for storyboards

For each storyboard frame:
  1. Generate or upload a sketch or hero still for the frame
  2. Image-to-video with simple motion ("subtle zoom" / "slow pan")
  3. Concat clips in After Effects or ffmpeg

Output is an animatic — not a final cut, but a directional reference for the live-action shoot or final animation.

Pattern: Social pre-roll variants

1. Approved hero image (1:1 or 16:9)
2. Generate 5 variants with different motion:
     - Subtle zoom in
     - Slow pan left to right
     - Tilt up revealing more of the scene
     - Static with subtle parallax
     - Dolly forward
3. A/B test on social

Five short clips from one hero image. The cost is fixed (5 generation calls), the output is varied enough for meaningful A/B testing.

Rate Limits and Cost

Video generation is more expensive than image generation in compute time and quota:

Plan video workloads with longer polling intervals (5-15s), wider rate-limit headroom, and async webhooks if available. Treat each video job as a multi-minute commitment, not a request-response.

IP Safety — The Differentiator

Firefly Video is designed to be commercially safe. Adobe states it is trained on:

• Licensed Adobe Stock content
• Public-domain content
• Openly-licensed content

For commercial campaigns, the commercial-safety posture is the deciding factor: the model's training-data provenance is what lets enterprise legal teams approve generated video for production use. IP indemnification for enterprise customers is defined in the customer's Adobe agreement — confirm scope with the account team and legal.

Validate

A Firefly Video pipeline is production-ready when:

1. Customer has the Firefly Video entitlement (verify SKU before building)
2. Generation jobs are tracked with appropriate timeouts (10-15 min max)
3. Output MP4s are downloaded and re-hosted within the URL expiry window
4. Prompts follow the structured pattern (subject + setting + motion + camera + style)
5. Use case is appropriate for clip-length output (not long-form)
6. Legal sign-off acknowledges IP indemnification posture

Troubleshooting & Edge Cases

• Output has incoherent or "morphing" subjects: Prompt is too complex. Simplify to one subject + one motion.
• Output is shorter than expected: clips can come back shorter than requested when temporal coherence degrades. Reduce motion complexity.
• Job stuck for 10+ minutes: Cancel and resubmit. Typical jobs finish in 1-3 minutes; long stalls are rare but happen during peak load.
• sizes rejected: sizes must be an array of {width, height} objects — the most common rejection is passing a single object instead of an array. Stick to the tested set (1920×1080, 1080×1920, 960×960, 1280×720, 720×1280) for predictable results; the API has accepted and completed other dimensions in live checks (e.g. 1080×1080), so an unusual size is not automatically the cause of a rejection.
• Request runs against the wrong model: Ensure the x-model-version: video1_standard header is present on the submission.
• Content safety filter triggers on a clean prompt: In practice, video prompts appear to trigger safety filters more readily than equivalent image prompts. Strip any reference to people, brands, or sensitive themes and try again.
• Image-to-video output ignores the source: Source image may not match the target aspect ratio. Pre-process to the exact target size.

Chaining with Other Skills

• firefly-services-storage-refs — Source image upload for image-to-video
• firefly-generate-image-v3-async — Generate the source image first
• firefly-services-rate-limits — Video quota is lower; plan capacity
• firefly-services-troubleshoot — Errors are usually safety-filter or rate-limit

References

• Firefly Video Model — Adobe announcement
• Firefly API Documentation Hub
• Firefly Services Trust and Safety