Checking a video

How does the agent see the video?

A coding agent can’t watch a video. So explainroo turns the video into things the agent can look at. There are still pictures, contact sheets and reports. The same tools help you when you change a video yourself.

How do I see single frames?

npx --yes explainroo still videos/dns                 # the end of each scene
npx --yes explainroo still videos/dns intro@2.5 12.0  # 2.5 s into "intro", and 12 s into the video

The PNG files land in out/stills/. A time can be written in a few ways. A plain number is seconds into the video. scene or scene@end is the moment a scene is fully built. scene@start is its first moment. scene@2.4 is 2.4 seconds into the scene.

What is a contact sheet?

A contact sheet puts many frames of the video on one picture. Under each frame it prints the scene and the time.

npx --yes explainroo sheet videos/dns                  # the whole video
npx --yes explainroo sheet videos/dns resolver         # one scene over time

The sheet of one scene is out/sheet-resolver.jpg. --large makes the frames 640 pixels wide, three to a row.

With one look you can see whether something new appears every few seconds, and whether a scene builds up in the right order. A frame caught while the camera moves, or while something draws itself in, has “camera moving” or “drawing in” under it. still prints the same next to its file. Such a frame can look cut off or empty and still be fine, so look at a moment after it.

To compare a few moments side by side, name them:

npx --yes explainroo sheet videos/dns --times 3,intro@2.5,40

This writes out/sheet-times.jpg with only those moments, in that order. It takes every time that still takes.

What does check find?

npx --yes explainroo check videos/dns

check draws every scene at many points in time. It looks at text and math formulas, and the label of an arrow counts as text. It finds text that is:

• off the frame or too close to its edge
• under the captions, or where a platform’s buttons go
• under the watermark, also where the zoom of the scene’s own s.camera puts it there
• on top of other text, or partly over a shape, like a note on the edge of a box
• sitting on the line drawn around a box, more than 4 pixels into it
• crossed by the line of an arrow or by a mark from s.annotate
• cut at the edge of the frame by the camera while a shot holds. A held shot of a table that shows some rows without the header counts as cut
• too small at the size the camera shows it. In a tall or square video, which people watch on a phone, text below 40 pixels at 1080p gets a hint too
• too close in color to what is behind it, like the page or its own card. A background painted with s.bg counts too

It also finds these problems:

• shapes that partly cover each other, like two cards. overlap: true on a shape says it is meant
• an arrow whose line runs through a shape that is not one of its ends, or runs off the frame
• an arrow that points at nothing for a while, because it comes in before the element at one of its ends
• a sticker from s.short that covers a picture or text. overlap: true lets a sticker cover only backgrounds
• a mark from s.annotate that stays on screen more than 0.3 seconds after the element it marks has left
• a code window or a terminal that is empty for more than a second before its first line
• a terminal command written with the prompt, like { cmd: '$ npm test' }, which shows $ $ npm test
• text that types or writes itself, like typed code or a terminal, when it isn’t all there by the end of its scene. A number that counts up gets the same check
• a scene that shows only its background for more than 1.5 seconds, at the start or later, when everything has left and nothing new has come yet. This is a hint
• a cover that shows only text on an empty background, or an image that fit: 'cover' cuts, which can cut off words or a face
• a first frame that shows only the background, or only elements still on their way in. Many services use the first frame as the thumbnail. The hint names an element to give at: -1, so it is there from the first frame
• text of two scenes in the same place during a zoom transition between them. This is a hint, and {transition=push} on the scene moves the two scenes side by side instead
• a text that shows four or more words in a row that the captions show at the same moment. This is a hint
• a recording of an app whose page text would play too small. The warning names a viewport that fits
• text with a background but no words
• code lines that are wider than their window, in every code step
• math formulas that change over time or are called only after a scene starts
• missing clips and sound files, and invalid clip options
• clips that freeze on their last frame for more than 0.3 seconds
• chapters that are too few or too short for YouTube, in a wide video. These are hints
• elements that are timed after their scene has ended
• a cue word that is spoken more than once. This is a hint, and 'word#2' picks the second time
• errors in scenes.js, like a word cue that is not in the narration or an icon name that doesn’t exist. It also lists close matches. After a missing cue, it checks the rest of the scene as if the cue were at the scene start, so one run shows everything
• words the voice said differently from the script, with the words around them in what it heard
• a video that runs longer than its targetSeconds
• long stretches where nothing new appears. This one is only a hint

When a color is hard to read, check suggests a readable one and gives its contrast ratio. It suggests a darker or lighter shade of the same color first, so the color keeps its meaning, and the look’s ink second. It also checks a custom look’s ink and muted against its background.

A finding names the element it is about by what it shows, like the box “Your phone”, and by its id when it has one. An element with neither is named by its call and its place, like “the 2nd s.draw at 960,735”. So give an element an id when a scene has several of the same kind.

Fix errors and warnings before you render. Hints are up to you. check does not see what a drawing made with s.draw or on s.ctx holds. It only knows that something is there. Give s.draw a box, and check treats that box as a shape, for overlaps and the edges of the frame. It also cannot read text inside a clip or an image, and it cannot tell whether clipped UI text is hidden. Look at stills for those, and for UI details during camera zooms.

Which words did the speech check accept?

The speech check listens to the narration and compares it with the script. Some words it hears a little differently, and it still counts them. check lists them so you can look:

• words that only nearly matched, like “Soundroo” heard as “Soundru”. Names the speech model doesn’t know often end up here
• words that sound the same, like “peace” for “piece”
• two words heard as one, like “Seacosts” for “See costs”, and the parts of a word with hyphens heard apart, like “well known” for “well-known”

These count as understood. check and verify give each one its time in the video, and it needs no fix unless it sounds wrong when you listen at that time. When there are more than two, they name them without their times, and --verbose adds the times. {shown|spoken} can’t change how they are heard. Narration made with an older version gets today’s speech check once, and the result is saved.

A word the check could not confirm is different. voice quotes the sentence the voice was given, after {shown|spoken}, and what the speech model heard there, with four words on each side. Then it says what to do.

Sometimes a word is said right but heard as another spelling, like “Reg X 101” for regex101. A "heard" line in video.json accepts that form, and the word then counts as confirmed in every check. The voice isn’t made again. The hint of the speech check names the line to add. See the speech check.

In six languages, the speech check also reads spelled numbers as digits, and an amount like “cem reais” as R$ 100. See numbers and amounts.

How do I check a smaller player?

A video that looks clear full size may be hard to read in a feed. Check the width viewers will see:

npx --yes explainroo check videos/dns --view-width 854
npx --yes explainroo still videos/dns --scale 0.4448

The first warns when visible text shrinks below 16 pixels at that width. The second saves stills at about 480p for a wide 1080p video. Use bigger text or zoom into the part that matters. This checks size, so also watch the platform’s lower quality copy after uploading.

How do I check several sizes?

Run check and look at a sheet for each size:

npx --yes explainroo check videos/dns --size tiktok
npx --yes explainroo sheet videos/dns --size tiktok

A wide layout may need to stack its elements in a tall frame. See publishing a video.

To look at what a package for a platform will show, use --platform instead of --size. It also takes that size’s block in "sizes" and the platform’s watermark rule, so --platform tiktok has no watermark:

npx --yes explainroo check videos/dns --platform tiktok
npx --yes explainroo still videos/dns --platform tiktok

What does verify check?

npx --yes explainroo render videos/dns
npx --yes explainroo verify videos/dns

verify looks at the finished MP4, WebM or MOV. It measures how loud it is and how high the peaks go. It looks for black frames and long silence. It also listens to the finished sound to make sure the voice is still clear over the music and the sound effects. It saves what it found in out/report.json.

render keeps the true peak of the sound at -1 dBTP or lower. verify warns when a file goes above that, and it fails when it goes above 0 dBTP, where the sound clips. It also lists the words that only nearly matched, next to the share of the narration it understood. Each word it could not confirm is a warning with its time in the video.

The last line of verify is its verdict. It says “passed”, or “passed with 2 warnings” when there are warnings. When something fails, it says “failed” with the number of problems, like “failed: 1 problem”. With --json, the result has verdict, and passed is true or false.

For another size, give verify the same flag you gave render, like verify videos/dns --size tiktok or --platform tiktok. It then checks the file that render wrote with that flag. --file picks any other file. Transparent videos get no black-frame check. Use still --transparent to look at their PNG frames. verify does not check GIFs.

Can I check a video made with another tool?

Yes. check and verify also take a video file instead of a project, made with explainroo or with any other tool. They take the common video files, like MP4 and MOV, and both give the same report:

• black frames, and blank frames of one other color
• stretches where the picture stands still for 3 seconds or more
• near silence of 2.5 seconds or more
• the loudness against -14 LUFS, and the true peak against -1 dBTP
• text too small to read on a phone

npx --yes explainroo check talk.mp4 --script script.txt

With --script, the speech model compares the speech with that text and names the words it didn’t hear. An explainroo script.md works too. Add --language es for another language. The exit code is 1 when the video failed, so a script can stop on it.

Text counts as too small under 2.2% of the frame’s short side, which is 24 pixels at 1080p. In a tall or square frame, text under 40 pixels at 1080p gets a hint. check looks at a frame every second, or at 120 frames in a video longer than 2 minutes, and --step sets the seconds. It finds the text from the pixels, so its size is about right, not exact. In a video that isn’t an explainroo project, small text is only a hint, because a clock or a terminal in a recording may be fine as it is.

still and sheet take a video file too, also a phone video shot upright. Times are in seconds:

npx --yes explainroo still talk.mp4 3 12.5
npx --yes explainroo sheet talk.mp4

The first writes talk.stills/t3.00.png and t12.50.png. Without times, it writes three frames spread from the start to the end. The second writes talk.sheet.jpg, with a frame every few seconds and its time under it. It takes --every and --times 3,12.5 like the sheet of a project.

To cut a recording of your own by its words, see cutting your own recording.

Can I watch it while I edit?

npx --yes explainroo preview videos/dns

preview opens a page in your browser. It shows the video, a timeline and the list of scenes. When you save script.md or scenes.js, the page reloads. Space plays and pauses. The left and right arrow keys jump one second. Comma and period go one frame back or forward. The square brackets jump to the scene before or after.

render --draft makes a quick video at half size. Use it when you want to watch the video with sound before the final render.