Lecture video: beta acceptance tests

A new lecture video reached Bunny and played with English captions. Admin model saves, resets and stale-write rejection also passed.

SP-679 | Live tests: 16 November 2026 | Report: 17 November 2026

Status as of 17 November 2026

Core beta flow passed. Full acceptance remains incomplete. Transcript persistence, direct invalid-ID API writes, saved-model retry checks and live provider faults still need verification.

These screenshots and this generated video come from the real beta LMS, using a synthetic three-scene lesson in an isolated QA course. This is not a local mock demonstration.

PR #622, merged | Successful staging admin deployment | Jira SP-679

Why these checks matter

The admin and runner were blocked before reaching backend authentication. After the authorized Cloudflare configuration change and beta routing updates, the admin could manage model choices and the runner could complete a fresh generation. Delivery, captions and playback were checked separately.

Upload

Synthetic XLSX and three-slide PPTX.

→

Review

English detected. Instructor confirmed US English.

→

Generate

Six speech beats and a new Bunny delivery.

→

Play

22.87 seconds, captions and seeking checked.

The new generated video

This is the 480p stream exported from the newly delivered Bunny video. It contains the generated narration. It is output evidence, not a recording of the complete UI test session.

Live player proof

New beta video paused on the Explain the idea slide with matching English captions.
Actual Chrome capture from beta. Seeking to the middle of the new video displays the matching English caption over the slide.

Acceptance checks and results

Live observations, local automated checks and unverified cases are listed separately.

CheckResultEvidence and scope
Admin accessLive passConfig loads through the Cloudflare beta route.
Catalog default and choicesLive passOpus 5.5 is the default. The editor offers the two approved exact IDs.
Global save and resetLive passSaved Opus 5, saw the global override, then reset to Opus 5.5.
Tenant override and resetLive passSaved a tenant override. Reset preview showed the inherited model. The tenant returned to Default / Opus 5.5.
Stale editorLive passA second tab saved a newer revision. The older tab was rejected with HTTP 409.
Anonymous and invalid sessionsLive passBoth direct staging-admin catalog probes returned HTTP 401.
Upload, profile and language reviewLive passThree-scene XLSX/PPTX accepted. English detected. The job waited for instructor confirmation before audio.
New Bunny uploadLive passNew job delivered successfully. The resumed Cloud Run execution completed in 93 seconds.
New captionsLive passEnglish WebVTT has six cues matching the synthetic script. The instructor list reports captions attached.
Playback and seekingLive passChrome played to the end of the 22.87-second video. Seeking changed the slide and displayed the matching English caption.
Reloaded delivery stateLive passDelivered and captions-attached state remained after a full page reload.
Failed script profilingLive passA one-scene negative fixture failed the minimum-scene self-check after four bounded attempts. The UI showed failure and retry; the course slot was released.
Transcript database persistenceNot verifiedThe direct beta transcript read was not completed. A saved caption track alone is not independent proof of the transcript database row.
Provider denial, retirement and outageAutomated passFive focused runner tests cover contract validation, legacy evidence, model redirection/retirement, credential denial, retry bounds and deadline expiry. Provider responses are mocked.
Backend model contractsAutomated pass25 backend unit tests passed. These are local tests, not live beta fault injection.
Invalid model IDs through live APINot verifiedThe live dropdown restricts choices. Direct authenticated invalid-ID submissions were not completed.
Saved model through edits/retriesNot verified liveThe current report does not claim a live inspection of the saved job model or a retry after a configuration edit.
Production generationNot runNo production render or production admin backend change was made.

Repeat the remaining tests

  1. Open the QA course and inspect the new video's transcript through the beta application API or transcript editor. Confirm language, origin, segment count and persistence after reload.
  2. Use authenticated admin API requests to submit unsupported IDs, wrong case, whitespace and aliases. Expect rejection with the saved setting unchanged.
  3. Confirm a job, change the tenant model, and verify that the original job and its retry retain the original saved model. Start a new job to check the new effective model.
  4. Run isolated provider-failure tests without changing credentials for other beta users. Verify that failed preflight starts no speech or rendering.

Known limits and follow-up

Evidence records

Successful job: TozDf_OA-KwLbo_wRoWpg. Video: c5622021-803e-4a6d-ba43-903bfb3a9e4a. Final execution: lecture-video-runner-beta-4sdr6.

The exported MP4 fully decoded without errors. It is 1,661,875 bytes, 854 x 480, H.264/AAC. The English VTT is 568 bytes with six cues.

Download generated video | Download English captions | Structured results

How to use the feature

  1. In staging admin, open Config and search for lecture. The model editor offers the approved choices and shows the source of the effective value.
  2. For a tenant override, open Tenant overrides from the model row. Reset previews the inherited value before removing the override.
  3. In the QA course, upload a script and slides. Confirm the narration language before generation continues.
  4. Wait for delivery and caption attachment, then check playback. Caption attachment can finish after the delivered status first appears.