Using Marengo 3.0 in Amazon Bedrock to Search Video by Natural Language
According to the Amazon Machine Learning blog, TwelveLabs Marengo Embed 3.0 is now a selectable embedding model in Amazon Bedrock Knowledge Bases, turning raw video, audio, and image files into searchable vectors. For anyone who has spent hours stitching together transcription services, frame‑extraction scripts, and separate vector stores, the managed offering promises a single‑click way to ask things like “show me the penalty kick in the second half” and get the exact clip.
Creating a Managed Knowledge Base in a Few Clicks
The console walk‑through is straightforward: upload an MP4 (the blog uses a 10‑minute FIFA final clip) to an S3 bucket, launch a Managed Knowledge Base, pick Marengo Embed 3.0 as the embedding model, and hit Sync. Behind the scenes Bedrock does three things automatically:
- Segmentation – chops the video into 4‑second chunks (the default for both audio and video).
- Transcription & Frame Sampling – extracts speech text and representative frames without any extra code.
- Embedding – feeds each chunk to Marengo 3.0, which outputs a 512‑dimensional vector that blends visual, textual, and acoustic signals.
When the sync finishes, the console’s Test tab lets you run a natural‑language query and shows the matching chunk timestamps, source URI, and even a playback widget.
How Marengo Embed 3.0 Packs Multimodal Information
Marengo 3.0 is a multimodal embedding model: it processes video pixels, audio waveforms, and any accompanying subtitles or OCR text in a single forward pass. The result lives in a compact 512‑dimensional space, meaning each segment occupies roughly 2 KB on disk (512 × 4 bytes). Because the model learns to align visual cues with spoken words, a query that mentions “penalty kick” can surface a clip even if the word never appears in the transcript but is visible on the field.
| Model | Modalities Covered | Vector Size | Default Segment Length |
|---|---|---|---|
| Titan Text (default) | Text only | – (text‑only) | N/A |
| Marengo Embed 3.0 | Video, Audio, Image, Text | 512 dims | 4 seconds (audio & video) |
The table highlights the trade‑off: you gain cross‑modal recall at the cost of larger indexes (every 4‑second slice becomes a row) and longer ingestion time.
What the Managed Pipeline Actually Trades Off
The headline benefit is no infrastructure. In practice that means you hand over control of segmentation length, chunk overlap, and even the transcription engine to Bedrock. If your use case needs finer granularity—say a 1‑second slice to catch rapid camera cuts—you must accept higher storage and retrieval latency, or build a custom pipeline outside the managed service. Likewise, the unified 512‑dimensional space is space‑efficient, but it is a single representation; you cannot query the visual and audio dimensions separately, which limits advanced filtering (e.g., “show me all clips where a whistle sound occurs but the ball is not visible”).
Another hidden cost is vendor lock‑in. The embeddings live in an Amazon‑hosted vector index that is only reachable via the Bedrock Retrieve API or the Bedrock AgentCore gateway. Exporting the vectors for use in another vector database is not a documented feature, so moving away later would require re‑ingesting the media.
Pricing Reality Check
Bedrock charges only for storage and model invocations. The blog does not give a per‑token or per‑embedding price for Marengo 3.0; it refers to the standard model‑invocation rate. In concrete terms, each 4‑second segment triggers one embedding call, so a 2‑hour video (7200 seconds) yields 1,800 embeddings. Multiply that by the invocation price you see in the Bedrock pricing page to estimate the cost. Storage is cheap—each 512‑dim vector is a few kilobytes—but the total size scales linearly with segment count, so very long archives can still add up.
Quick‑Start: Run a Semantic Video Search Today
- Create an AWS account (if you don’t have one) and enable Bedrock in a supported region (us‑east‑1 or eu‑west‑1).
- Upload a short video (under 5 minutes) to an S3 bucket.
- In the Bedrock console, choose Knowledge Bases → Create Managed KB.
- Select TwelveLabs/Marengo Embed 3.0 as the embedding model and point the data source to your bucket.
- Click Sync, wait for the ingestion job to finish, then open the Test tab.
- Type a simple query like “show me the goal” and note the returned timestamps.
You now have a working multimodal search without writing a single line of code. If you want to integrate it into an app, copy the Python snippet from the console’s Details page and call the Retrieve API with the same query string.
