Lecture video: beta acceptance tests
A new lecture video reached Bunny and played with English captions. Admin model saves, resets and stale-write rejection also passed.
SP-679 | Live tests: 16 and 17 November 2026 | Report: 17 November 2026
Status as of 17 November 2026
Requested beta follow-up checks passed. Invalid model IDs, persisted transcript metadata and saved-model behavior across a setting change and retry are now verified through the live API. Provider failure cases passed local mocked checks, as requested. This report does not claim exhaustive acceptance or production verification.
These screenshots and this generated video come from the real beta LMS, using a synthetic three-scene lesson in an isolated QA course. This is not a local mock demonstration.
PR #622, merged | Successful staging admin deployment | Jira SP-679
Why these checks matter
The admin and runner were blocked before reaching backend authentication. After the authorized Cloudflare configuration change and beta routing updates, the admin could manage model choices and the runner could complete a fresh generation. Delivery, captions and playback were checked separately.
Upload
Synthetic XLSX and three-slide PPTX.
Review
English detected. Instructor confirmed US English.
Generate
Six speech beats and a new Bunny delivery.
Play
22.87 seconds, captions and seeking checked.
The new generated video
This is the 480p stream exported from the newly delivered Bunny video. It contains the generated narration. It is output evidence, not a recording of the complete UI test session.
Live player proof

Acceptance checks and results
Live observations, local automated checks and unverified cases are listed separately.
| Check | Result | Evidence and scope |
|---|---|---|
| Admin access | Live pass | Config loads through the Cloudflare beta route. |
| Catalog default and choices | Live pass | Opus 5.5 is the default. The editor offers the two approved exact IDs. |
| Global save and reset | Live pass | Saved Opus 5, saw the global override, then reset to Opus 5.5. |
| Tenant override and reset | Live pass | Saved a tenant override. Reset preview showed the inherited model. The tenant returned to Default / Opus 5.5. |
| Stale editor | Live pass | A second tab saved a newer revision. The older tab was rejected with HTTP 409. |
| Anonymous and invalid sessions | Live pass | Both direct staging-admin catalog probes returned HTTP 401. |
| Upload, profile and language review | Live pass | Three-scene XLSX/PPTX accepted. English detected. The job waited for instructor confirmation before audio. |
| New Bunny upload | Live pass | New job delivered successfully. The resumed Cloud Run execution completed in 93 seconds. |
| New captions | Live pass | English WebVTT has six cues matching the synthetic script. The instructor list reports captions attached. |
| Playback and seeking | Live pass | Chrome played to the end of the 22.87-second video. Seeking changed the slide and displayed the matching English caption. |
| Reloaded delivery state | Live pass | Delivered and captions-attached state remained after a full page reload. |
| Failed script profiling | Live pass | A one-scene negative fixture failed the minimum-scene self-check after four bounded attempts. The UI showed failure and retry; the course slot was released. |
| Transcript database persistence | Live pass | Authenticated beta API returned transcript_ready=true, en-US, lecture_script and six segments. The same metadata and saved timestamp remained after a full page reload. |
| Provider denial, retirement and outage | Automated pass | Five existing runner tests passed again. Seven additional local mocked checks passed: 401 credentials, 403 access denial, 404 missing model, 429 rate limit, 529 outage, connection timeout and Retry-After beyond the deadline. Live provider fault injection is outside the agreed scope. |
| Backend model contracts | Automated pass | 25 backend unit tests passed. These are local tests, not live beta fault injection. |
| Invalid model IDs through live API | Live pass | Six authenticated tenant writes returned HTTP 400 video_model_invalid: alias, typo, wrong case, trailing whitespace, unknown ID and null. The inherited setting remained unchanged. |
| Saved model through edits/retries | Live API pass | Job A retained default Opus 5.5 after the tenant changed to Opus 5 and on retry. Newly confirmed job B received tenant Opus 5. A controlled worker read the backend snapshots and stopped before generation. Both QA jobs are terminal and the tenant override was removed. |
| Production generation | Not run | No production render or production admin backend change was made. |
Saved model lifecycle on beta
These checks used the real beta confirm, configuration, retry and internal worker APIs. A controlled test worker claimed the dedicated QA jobs before the automatically dispatched runner. It read the saved model and then reported an intentional test failure before provider calls, audio, rendering or Bunny upload. This is an API lifecycle test, not a second full video rendering test.
| Action | Observed saved model |
|---|---|
| Confirm job A with no override | claude-opus-5-5, source default |
| Change tenant setting to Opus 5; read job A again | claude-opus-5-5, source default |
| Retry job A | claude-opus-5-5, source default |
| Confirm new job B | claude-opus-5, source tenant |
| Cleanup | Both QA jobs stopped; tenant override removed; default Opus 5.5 restored. |

Provider failure test scope
As requested, credential denial, model access denial, rate limits, outages and timeouts are checked with local mocked responses. Credential and access failures make one request. Persistent rate limits, outages and connection timeouts stop after three requests. A Retry-After delay beyond the remaining deadline is rejected after one request. No shared beta provider credentials were changed.
Known limits and follow-up
- The browser connection was restored for the follow-up checks on 17 November. Provider fault injection on live beta is not required by the agreed test scope. Production generation remains untested.
- The beta callback URL was restored manually to beta.skillpixel.vn for these tests. PR #717 is open and makes the public beta and production routes permanent in the deployment workflow. Until it merges, another deployment can overwrite the manual beta repair. Production was not changed.
- Bot Fight Mode is off across the shared skillpixel.vn zone, as explicitly authorized. No other bot configuration changed. This zone setting also affects production traffic.
- The original global inheritance was preserved. The follow-up tenant override was deleted with a revision check. Both global and tenant model inheritance are restored to Default / claude-opus-5-5.
- One-scene profiling failed before audio. The valid three-scene input passed after a repair. Those are two distinct test jobs.
- Zero pending usage events proves acknowledgments, not invoice or database-ledger reconciliation.
Evidence records
Successful job: TozDf_OA-KwLbo_wRoWpg. Video: c5622021-803e-4a6d-ba43-903bfb3a9e4a. Final execution: lecture-video-runner-beta-4sdr6.
The exported MP4 fully decoded without errors. It is 1,661,875 bytes, 854 x 480, H.264/AAC. The English VTT is 568 bytes with six cues.
Download generated video | Download English captions | Structured results | Invalid-ID responses | Transcript persistence | Saved-model snapshots | Lifecycle cleanup | Provider unit checks
How to use the feature
- In staging admin, open Config and search for lecture. The model editor offers the approved choices and shows the source of the effective value.
- For a tenant override, open Tenant overrides from the model row. Reset previews the inherited value before removing the override.
- In the QA course, upload a script and slides. Confirm the narration language before generation continues.
- Wait for delivery and caption attachment, then check playback. Caption attachment can finish after the delivered status first appears.