Manual test plan

The full manual regression plan for Immersive, organized by user journey. Every section states its Preconditions & data (exact accounts, seed commands, console recipes), numbered Steps, Expected outcomes (including the FakeAI marker output that is the correct result in development), and Variants (locale / dark mode / mobile) where they matter.

This document subsumes the batch manual-testing checklists (most recently issue #530, the 2026-07-24 overnight batch): those checks are folded into the relevant sections below and tagged [#530-A] .. [#530-E]. Future batch issues should land here the same way.

Two appendices close the doc:

  • Appendix A — Release smoke test: the ~10-minute subset to run after every production deploy.
  • Appendix B — Pending manual actions (one-off ops): console/VM actions that are operations, not repeatable tests, each with its source PR/issue and a verification step.

Automated coverage (unit, integration, system tests, bin/ci) is assumed green before any of this runs — this plan is only for what needs human eyes, ears, or hardware.

0. Environment and test data setup

0.1 Environments

Environment URL / command Notes
Development bundle exec foreman start -f Procfile.dev → http://localhost:3000 Web + Sidekiq worker. AI calls faked by FakeAI unless LIVE_MODE=true.
Development, live APIs LIVE_MODE=true bundle exec foreman start -f Procfile.dev Real OpenAI/AWS calls. Requires working credentials (see 0.4).
Production https://immersive-app.com Deployed via bin/deploy (Kamal → Hetzner, behind Cloudflare).

0.2 Accounts

Development (bin/rails db:seed, all passwords immersive):

Email Role Languages (target ← fluent)
admin@example.com admin fr ← en
james@example.com learner (primary persona) fr ← en
juliet@example.com learner (second persona — peer flows) fr ← en
user10@example.com … user40@example.com learners fr ← en
user50@example.com … user90@example.com learners (reverse pair) en ← fr

Use james as the learner under test, juliet as the counterpart in any two-account flow (Evaluate peer grading, teacher grants, deck publishing), and admin for Section 10. The en ← fr users (user50@+) are the accounts for testing a non-English fluent language and the en target-language content.

Production: use your own admin account plus a disposable learner account per test run (sign up with a +-suffixed address); never grade/grant against a real user’s data.

0.3 Content seeding (development)

Run once per fresh database, in this order:

bin/rails db:seed                          # users above
bin/rails verbs:import_all                 # verb + conjugation corpus, all six languages
                                           #   (or verbs:import_french etc. for one language)
bin/rails grammar:seed_reading_phenomena   # EuRom5 phenomenon sections + notes (#412)
bin/rails immersive:seed_starter_decks     # Essentials starter decks (fr/es/pt/en)
bin/rails immersive:seed_library_documents # Document Library: 3 texts per language (#489, PR #525)
bin/rails "frequency_decks:import[fr,en]"  # frequency starter decks - LIVE_MODE=true + AWS
                                           #   Translate creds required; ~5,000 translate calls
                                           #   per pair. Skip unless testing import quality.

Notes:

  • immersive:seed_library_documents and the en starter deck arrived with PR #525 (July 2026); on checkouts older than that merge, those rows won’t exist in your dev DB.
  • Grammar notes content ships in the repo/DB via migrations and imports; grammar exercises seeding (grammar:seed_exercises[language,level]) needs a real LLM backend and is parked on #390 — expect zero exercises in a fresh dev DB.
  • All seed tasks are idempotent and safe to re-run.

0.4 FakeAI vs LIVE_MODE — what needs real keys

See docs/technical/fake_ai.md. In development every external AI call is faked by default; the fake output is the expected result for those tests. Marker forms:

Backend Faked in dev? Marker output you should see
LLM prompts (app/prompts: coach, writing correction, rewrite/tone/level-up, verb/sentence generators, transcription/OCR) Yes Text beginning Fake <label> response - set LIVE_MODE=true for real API calls. Writing correction returns your submission echoed with one marked correction. Schema’d prompts without a shaped fake surface as a handled LLM-error state — that error UI is the pass condition.
AWS Translate (word/sentence/title translation) Yes (<lang>) <original text> — e.g. (en) Le chat dort.
Sentence embeddings (usefulness ranking, #191) Yes No visible marker; ranking is deterministic per text.
AWS Polly (all resource audio, read-aloud, dictation) No Not covered by FakeAI. Without Polly credentials, audio creation fails (logged error, player does nothing). Any test that must hear audio needs LIVE_MODE=true + AWS creds, or production.

LIVE_MODE=true requirements per feature area:

  • Real OpenAI/LLM key: coach, writing assistant corrections, document rewrite/tone/level-up, sentence generation, image/PDF OCR import, grammar exercise generation.
  • AWS Translate: reader tap-translate quality, title gloss quality, frequency deck imports.
  • AWS Polly: everything audible — resource audio players, listen-while-reading, dictation, review-card audio.

When in doubt: functional wiring is tested in dev against FakeAI markers; content quality (translation accuracy, audio naturalness, LLM output) is only testable with LIVE_MODE or in production.

0.5 Getting FSRS cards into every state

Reviews schedule cards through FSRS (status: new → learning → review / relearning, plus suspended; due_at drives the queue). Two recipes:

Honest recipe (preferred for one deck) — review, then age the deck:

  1. As james, open a starter deck and review ~10 new cards, mixing ratings (Again / Hard / Good / Easy). Cards are now learning/review with near-future due_at.
  2. Age them from the console so they come due again:
# bin/rails console
user = User.find_by(email: "james@example.com")
deck = user.decks.order(:created_at).last          # or find_by(name: "...")
deck.cards.where.not(status: "new").update_all("due_at = due_at - interval '3 days', last_review_at = last_review_at - interval '3 days'")

Fast recipe (bulk states without reviewing) — fabricate aged cards:

# bin/rails console
user = User.find_by(email: "james@example.com")
deck = user.decks.order(:created_at).last          # or find_by(name: "...")
cards = deck.cards.order(:id)
cards.limit(5).update_all(status: "review",     due_at: 2.days.ago,   last_review_at: 12.days.ago, stability: 8.0, difficulty: 5.0, reps: 4, lapses: 0)   # overdue
cards.offset(5).limit(5).update_all(status: "review", due_at: 5.days.from_now, last_review_at: 1.day.ago, stability: 12.0, difficulty: 4.0, reps: 6) # not yet due
cards.offset(10).limit(3).update_all(status: "relearning", due_at: 1.hour.ago, last_review_at: 1.day.ago, stability: 1.0, difficulty: 7.5, reps: 5, lapses: 2)
cards.offset(13).limit(2).update_all(status: "suspended")
# everything else stays status "new", due_at NULL

The fast recipe skips creating matching reviews rows, so Review History (4.6) and the profile Progress charts stay honest only under the honest recipe — use fast for queue/session states, honest for history surfaces.

0.6 The six locales

UI locales: en, fr, es, it, pt, ca (config/locales/*.yml). The UI locale follows the user’s fluent language (session-persisted); anonymous visitors get the language picker / info/languages edit page. Section 12 defines which journeys get which locales.

1. Public / anonymous

Preconditions & data: signed out (private window). Dev needs verbs:import_all for the SEO surfaces. Production: nothing.

Steps

  1. Visit / (root = public verbs index). Browse the verb list for the default language.
  2. Open a verb show page; open one conjugation show page (the SEO verb/conjugation surfaces, #178): check breadcrumbs, heading hierarchy, internal links between conjugations, and a sane <title>/meta description (view source).
  3. Visit /about, /faq, /terms, /privacy, /cookies, /contact_us. Submit the contact form.
  4. Visit /anki: the free Anki deck download page; download one deck.
  5. Visit /grammar and a grammar note while signed out — confirm whatever is public renders without a crash and gated actions redirect to sign-in.
  6. Hit an authenticated URL (e.g. /profile) — redirected to sign-in.
  7. Sign-up flow: pre-signup page (/user_informations/new) → Devise registration → confirmation email (see Section 11) → confirmed sign-in.
  8. Sign-in wrong password 11 times quickly from one IP — Rack::Attack throttling kicks in (10/min per IP) with the styled 429 response.

Expected: all pages render with no layout breakage; contact form persists and shows the thank-you state; sign-up sends a confirmation mail; throttle returns the throttled responder page, not a raw error.

Variants: verb/conjugation SEO pages in at least fr + en; dark mode on the landing and one SEO page; mobile width on the landing page.

2. Onboarding

Preconditions & data: a brand-new account from Section 1 step 7 (or dev: sign up fresh — do not reuse james).

Steps

  1. After first sign-in, the language grid (/onboarding/languages) appears: confirm the grid lists the supported languages with fluent / target selection and level choices.
  2. Pick a pairing different from the sign-up defaults (e.g. target it, fluent en) and a level; confirm.
  3. Land in the app; check the chosen pair is live: decks/documents/verbs surfaces show it content, UI locale follows the fluent language.
  4. Revisit /onboarding/languages — it shows the saved state, editable.
  5. Change languages later from Profile → languages (PATCH /profile/languages) and confirm the app follows.

Expected: grid renders in the visitor’s locale; saved pairing persists across sign-out/in; no route into the app skips onboarding for an unconfirmed grid.

Variants: run once with fluent fr (locale should be French throughout onboarding); mobile width — grid must not overflow.

3. Documents

Preconditions & data: james (fr ← en). Library needs immersive:seed_library_documents (dev) / the production content run (Appendix B). For OCR import, have a photo/PDF of printed French text. Audible checks (3.6–3.8) need Polly (LIVE_MODE or production); everything else works against FakeAI markers. A document with >1 revision is created in step 3.10.

Steps

  1. Create/upload/import (/documents/new): (a) paste text; (b) upload an image/PDF → OCR transcription. In dev, (b) yields the FakeAI completion block (Fake ... response + two French sentences) — that marker text becoming the document is the pass.
  2. Processing: the document is segmented into sentences/words (Sidekiq must be running); status reaches readable, level label sane.
  3. Reader + comprehension aids: open the document. Tap a word — the aid bar shows the gloss/translation (dev: (en) <word>); tap a sentence — sentence translation (dev: (en) <sentence>). The EuRom5 aid layer (#411) shows per-word aids and any linked reading-phenomenon cross-links (#412). Title gloss: the translated title renders under the original.
  4. Inference cards: from a tapped word, add a card to a deck (inference card flow) — confirm it appears in the target deck.
  5. Coach: run the comprehension coach — dev expects the FakeAI response or the handled LLM-error state.
  6. Read-aloud / listen-while-reading [#530-A] (needs Polly): tap Read to me — audio plays, the current sentence highlights and follows, tapping another sentence jumps playback there. Pause/resume; 0.7× slow audibly slows; dismissing restores the aid bar’s last-word display. Change voice in the sheet — next playback uses it and the choice is sticky per language.
  7. Dictation (needs Polly): start dictation from the reader; audio plays per sentence, typed answers are checked, progress advances.
  8. Writing assistant: open a writing document; submit text to the assistant panel (writing_submissions). Dev: the shaped fake echoes your text with one marked correction — apply it and confirm the document updates. Check the submissions list and show pages.
  9. Rewrite / tone / level-up / export: run rewrite, tone change, and level-up (dev: FakeAI marker or handled-error state). Export DOCX — file downloads and opens.
  10. Revisions: edit the document text twice; open Revisions — both listed with diffs; restore an older one; content reverts and the restore itself is a new revision.
  11. Topics: assign a topic, confirm it displays and filters on the index; clear it.
  12. Library copy-on-write [#530-B]: open /documents/library — 3 texts per language; open one — reads correctly, level label sane. Tap a word (aid works read-only), then make any edit — a personal copy is created; the library original is unchanged (verify from juliet).
  13. Bookmarks: bookmark a document; check /bookmarked_documents.

Expected: per step above. No FakeAI marker text should ever appear in production — seeing one in prod is a failure.

Variants: reader in dark mode (aid bar + follow-along highlight must stay visible [#530-A]); mobile Safari for 3.6 (transport usable); repeat 3.3 with an en ← fr user (user50@) to confirm aids translate into French.

4. Decks and reviews

Preconditions & data: james, with starter decks (immersive:seed_starter_decks, optionally a frequency deck) and the FSRS state recipe from 0.5 applied to one deck. For the bookmark import, bookmark a few words/sentences first (reader taps or verb pages). For spreadsheet import, download the template from the import screen and fill 5 rows. Anki export needs a working mail path (Section 11) — dev uses letter_opener/log unless SMTP configured.

Steps

  1. Workspace: /decks — folders, deck rows, due counts. Create a deck; create a folder; move the deck into it; rename; delete an empty deck. Copy a deck; publish one and find it under /decks/public from juliet’s account; copy the public deck as juliet.
  2. Card types: in one deck create each type — word card, sentence card, number card, text card, verb card. For verb cards use the all-persons-on-one-card style (#458): one card carrying all six persons of a tense. Edit a word card: change text + example sentence, save, reopen — persisted; blank the example sentence — actually cleared. Edit a sentence card: change text, save — persists [#530-A].
  3. Example sentences on cards (#459): on a workspace card row, open the example flow — attach from documents, generate (dev: FakeAI sentence lines), or write manually.
  4. Imports: (a) spreadsheet import — preview shows parsed rows with errors flagged, then commit; (b) bookmark import — bookmarked words/sentences become cards; (c) Anki export — request export, receive the .apkg by email, import it into the Anki desktop app and spot-check cards render (front/back/audio fields).
  5. Review session states (deck prepared per 0.5): start a review — queue serves overdue first, then due, then new. Verify each mode: read, write, listen (needs Polly), speak. Auto-graded modes are two-step (answer → scored confirm → rating). Mode choice is sticky (leave, return — same mode preselected). Rate cards across Again/Hard/Good/Easy; confirm an Again card comes back within the session (relearning), suspended cards never appear, and the session completes with the summary screen. Revisit flow: from the summary, revisit Again/Hard cards — same mode preselected.
  6. Stats and history: deck review/stats renders; /decks/reviews/ history lists past sessions with per-session detail (honest-recipe data only, see 0.5).
  7. Chromeless check [#530-A]: during a review session the five-tab bar is absent; card forms/imports close back to the deck; deck detail has no stray close button.

Expected: FSRS scheduling is visible (intervals grow with Good/Easy, Again shortens); no card belonging to another user is ever reachable (spot-check by URL-editing a card id from juliet’s deck — 404).

Variants: one full review session on mobile width and one in dark mode; review a listen-mode session in production for real audio.

5. Verbs and conjugations (signed-in workspace)

Preconditions & data: james; verbs:import_all (or at least import_french). Audio checks need Polly.

Steps

  1. /verbs signed in: search/filter the list; open a verb.
  2. Verb show: conjugation tables per mood/tense; audio players on conjugations (background PrepareAudioJob warms audio after first visit — second visit should play instantly).
  3. Conjugation show page: example sentences, matched sentences (sentence-matching surfaces); generate example sentences (dev: FakeAI lines); delete one.
  4. Bridge card: from a conjugation, create a bridge card into a deck (bridge_card) — verify it appears in the deck.
  5. Validation surfaces: check a verb known to have data issues renders its validation states rather than crashing (see docs/features/verbs/verb_validator.md for what the validator flags); admin-side validator views are in Section 10.

Expected: tables complete for fr; audio icon always present, playing or synchronously creating audio on tap (with Polly); graceful error without it.

Variants: one verb page per language (six total) — data coverage differs by design (see the starter-content tracker); confirm sparse languages (ca) degrade gracefully rather than erroring.

6. Grammar

Preconditions & data: james; grammar notes present (fresh DB has them per the content tracker: ~300+ per language); grammar:seed_reading_phenomena run. Exercises: expect zero until #390 lands (LLM backend) — an empty/absent exercise block is currently the pass. Note the /grammar surfaces are still in the design loop (#469): visual polish is not under test, function is.

Steps

  1. /grammar: notes index renders, grouped/filterable by category.
  2. /grammar_categories and a category show page.
  3. Open a grammar note: content renders (markdown), bookmark it, confirm under bookmarks; unbookmark.
  4. Recommendations: wherever the app recommends notes (reader cross-links from reading phenomena #412, deck/review surfaces), follow one link end-to-end.
  5. Exercises (when #390 lands): open a note with exercises, submit a right and a wrong answer to check — graded inline.

Expected: no missing-translation keys in any of the six locales’ notes surfaces; bookmarking round-trips.

Variants: one note per language; dark mode on a note page [#530-A] — the modernized note styles (#517) must hold in both themes.

7. Profile

Preconditions & data: james with review history (honest recipe, 0.5) and a few documents; juliet signed in on a second browser/profile for the two-account flows. Invitations use devise_invitable — dev mail via log/letter_opener.

Steps

  1. /profile: reopens the last-used sub-tab (Sticky Setting) — visit Progress, sign out/in, /profile returns to Progress.
  2. Progress: charts/stats reflect the seeded review history and document activity.
  3. Activity: recent activity feed sane.
  4. Vocabulary: word list renders; bulk action (profile/vocabulary/bulk) works on a selection.
  5. Evaluate (#460): as james, get my code (/profile/evaluate/code) — token generated. As juliet, enter james’s code (/profile/evaluate/enter), open the grade form, submit a peer grade. As james, see the received grade on the Evaluate hub. Try an expired/garbage code — friendly rejection.
  6. Teachers (#460): as james, create a grant (/profile/teachers/grants/new) scoped to a subset of tabs. As juliet, claim it at /teach/claim; browse /teach/:grant_id progress/activity/vocabulary — read-only, and only the granted tabs; a non-granted tab URL is refused. As james, revoke the grant — juliet’s access dies immediately.
  7. Invite a friend: send an invitation; open the invite link from the email in a private window; complete account creation.
  8. Account (/profile/account): update name, gender, location; upload a profile image; remove it; change password; language pair change (profile/languages). GDPR export request (/gdpr_exports) — export arrives by email. Daily-review email toggle and the signed unsubscribe link (Section 11 step 4).
  9. Sticky settings spot-check (docs/features/sticky_settings.md): review mode (4.5), profile sub-tab (7.1), reader voice (3.6) — each persists with no explicit save.
  10. Chats sub-tab: not present (blocked on #300) — its absence is the pass.

Expected: peer grades and grants are strictly code-mediated — no enumeration of other users anywhere; /teach claim throttle (30/10min per IP) rejects a brute-force loop.

Variants: profile in mobile width (tab bar present [#530-A]); Evaluate flow once in a non-en locale pair.

8. Five-tab shell, chromeless surfaces, dark mode, mobile web

Preconditions & data: james; phone-sized viewport (real device or devtools emulation). [#530-A] throughout.

Steps

  1. Phone width: the persistent five-tab bar (Decks, Documents, Verbs, Grammar, Profile) is present on all five root surfaces and their children.
  2. Chromeless surfaces: the reader (3.3) and the review session (4.5) hide the tab bar; leaving them restores it.
  3. No doubled navbar or modal weirdness anywhere: open a modal (card form), a sheet (voice picker), and a Turbo-frame edit on phone width.
  4. Dark mode sweep: toggle OS dark mode; spot an article/info page, a modal, a grammar note, the reader, a review card — tokens hold, no unreadable text (#515/#517).
  5. Focus rings: tab through a form page — focus visibly rings (intended change, not a regression).
  6. PWA: check the manifest loads (app/views/pwa), add to home screen on a phone, launch — app opens and navigates; after a deploy, confirm no stale-asset weirdness (hard-reload once if the served assets look old — see caching notes in docs/technical/caching.md).

Expected: layout integrity at 360px width on every surface touched in Sections 1–7; no horizontal scroll.

Variants: light + dark on every step here by definition.

9. iOS shell (Hotwire Native)

Preconditions & data: a Mac with full Xcode; repo checked out. [#530-C]. Path configuration is served from /configurations/ios_v1.json.

Steps

  1. cd ios && xcodegen generate, open the project, ⌘R into the simulator.
  2. App boots against production; five native tabs render.
  3. Web chrome is hidden inside the shell: the d-hotwire-native-* class suppression applies under the native user agent — no doubled web tab bar, no web navbar.
  4. Reader and review session render chromeless in the shell.
  5. Sign in, review one card, open one document, play one audio.
  6. Known open follow-ups (#521): tab-title localization and canonical prod host — do not fail the run on these; note drift.

Expected: no white flashes or dead back-stacks navigating tab-to-tab; audio plays through the native shell.

Variants: dark mode in the simulator; one phone + one iPad simulator size.

10. Admin

Preconditions & data: admin@example.com (dev) / your prod admin. Non-admin negative check: james hitting /admin gets refused.

Steps

  1. ActiveAdmin dashboard loads; walk each menu section: Users (+ abilities, locking/banning per docs/features/users/locking_and_banning.md), Decks, Resources (documents, verbs, sentences, words), Study events, Ahoy events, Mailersend events, Agents.
  2. Impersonate a user from admin; act as them; deimpersonate.
  3. Sidekiq web UI at /admin/sidekiq (admin-authenticated): queues draining, no growing retry/dead sets.
  4. Design system pages: /admin/design_system index and each section page render (tokens, components) in light + dark.
  5. Reading phenomena (#412/#524): the phenomena sections/notes admin surfaces list the seeded EuRom5 inventory; edit one and confirm PaperTrail records the version.
  6. Knowledge base (#190, PR #529 — once merged): Knowledge Base menu → Domains (4 seeded), Sources, Insights CRUD; tag filtering; draft → reviewed transitions; non-admin refused.
  7. Verb validator/admin verb tooling: run the validator view on one language; flagged rows render.

Expected: every index paginates without error on production data volumes; no admin action 500s; PaperTrail versions visible where documented.

Variants: none (admin is desktop/en-only by convention).

11. Emails

Preconditions & data: production (MailerSend) or dev with letter_opener/log inspection. A real inbox you control. See docs/features/email.md, docs/technical/production/mailersend.md.

Steps

  1. Devise: trigger and open — confirmation (sign-up), password reset, invitation (devise_invitable, 7.7). Links resolve to the right host and work.
  2. Daily review email (#378): force-send from console to james with due cards: Users::DailyReviewMailer.digest(user_id: User.find_by(email: "james@example.com").id).deliver_now (silently sends nothing when no deck has due cards). Open it — due counts correct, deck links deep into the app.
  3. Anki export mailer: from 4.4(c) — attachment arrives, imports into Anki. (Known: 1 pre-existing Brakeman warning on this mailer — not under test.)
  4. Unsubscribe: the daily email’s unsubscribe link — GET shows the confirm page without a session; POST (and the RFC 8058 one-click) unsubscribes; subsequent daily mail stops.
  5. GDPR export mailer: from 7.8.
  6. Contact mailer: from 1.3, the admin notification arrives.
  7. MailerSend webhook + admin events: after the above, the Mailersend admin section shows delivery events; spot one bounce, if available, is recorded.

Expected: all mail renders in both a desktop client and a phone client; sender/DKIM alignment good (no spam folder); every link https to the canonical host.

Variants: daily review email in a non-en fluent locale (fr user).

12. i18n spot-matrix

Preconditions & data: one account per fluent language, or flip james’s pair via /profile/languages. bundle exec i18n-tasks health is green in CI — this section is for rendered checks only: truncation, overflow, mistranslation-by-context, missing interpolation.

Journey en fr es it pt ca
1. Public landing + SEO verb page x x x x x x
2. Onboarding grid x x       x
3. Reader + aids x x     x  
4. Review session (one mode) x x   x    
7. Profile tabs x x        
11. Daily review email x x        

(Every cell = run that journey’s happy path in that UI locale. The matrix rotates: next full pass, shift the non-en/fr checks to the languages skipped last time.)

Expected: zero raw i18n keys on screen; no button/label overflow at mobile width in the longest-string locales (fr, ca).

13. Cross-cutting

Preconditions & data: any signed-in account; production access for Sentry/Cloudflare checks.

Steps

  1. Error pages: force a 404 (/nope), and check the static pages render styled: 400.html, 404.html, 406-unsupported-browser, 422.html, 500.html.
  2. Rate limiting (docs/technical/rate_limiting.md): exercise one throttle beyond login (e.g. audio API: >30 audio creations/min as one user → 429 with Retry-After; contact form 5/hr). Confirm the throttle log line ([Rack::Attack] throttled ...) appears with a real client IP, not a Cloudflare IP (#505).
  3. Sentry: raise a test error in production console (Sentry.capture_message("manual test plan check")) — event arrives in the Sentry project; uptime monitor for /up is green.
  4. Health: /up returns 200 through the edge.
  5. PWA/caching quirks: after a deploy, hard-refresh vs normal refresh — no stale CSS/JS (Propshaft digests); check one fingerprint asset URL 200s.
  6. Audit trail: edit a versioned record (document, grammar note via admin) and confirm PaperTrail versions; confirm Ahoy events row count grows during a browsing session (admin → Ahoy).

Expected: nothing in this section should page anyone — these are observability confirmations.

Appendix A — Release smoke test (~10 minutes)

Run after every production deploy, in this order, as a real learner account (plus admin for step 8). Any failure = stop and assess before announcing the deploy.

  1. /up returns 200; Sentry has no new release-tagged error burst.
  2. Sign in. Five-tab bar present on phone width; dark mode sane on the decks index.
  3. Open a document: tap a word (aid + translation appears), Read to me plays real Polly audio with follow-along highlight [#530-A].
  4. Review 3 cards in a due deck (read mode): grading advances, summary reachable; no FakeAI marker text anywhere.
  5. Open a verb → conjugation page; play one audio.
  6. Open a grammar note.
  7. Profile → Progress renders; sticky sub-tab returns on revisit.
  8. Admin: dashboard + Sidekiq queues draining; Mailersend events flowing.
  9. Trigger one transactional mail (password reset on a test account) — arrives.
  10. Public check signed out: landing page + one SEO conjugation page + /anki download link.

After content-task runs (not every deploy) [#530-B]: starter decks visible for ca/it/en accounts, 5-card spot-check per new deck for translation quality; Document Library shows 3 texts per language; open one per language.

Appendix B — Pending manual actions (one-off ops)

Console/VM actions flagged in open PRs/issues. These are operations, not repeatable tests — each has a verification step, and once done and verified they should be checked off in their source issue and removed here. Recurring rituals (e.g. re-running content seeds when a language launches) live in the journey sections’ data recipes, not here.

B.1 Production content runs (PR #525 / #489, [#530-B])

On the VM after #525 deploys (one-shot, --roles web per the #379 lesson):

bin/rails immersive:seed_library_documents      # vendored texts; no API calls
bin/rails immersive:seed_starter_decks          # adds English Essentials; fr/es/pt skip
bin/rails "frequency_decks:import[ca,en]"
bin/rails "frequency_decks:import[it,en]"
bin/rails "frequency_decks:import[en,fr]"

Pre-check: the immersive-prd IAM user must hold translate:TranslateText — the #379 fr→en production run was blocked on exactly this; dev creds have it, prod unverified. Verify in the AWS console (IAM → immersive-prd → permissions) before the imports, or the first import fails cleanly.

Open decision for Sean: whether to also run the remaining pairs (frequency_decks:import_all), and the Catalan verb-corpus expansion (parked in #489).

Verify: Appendix A’s content-task block — starter decks for ca/it/en, 3 library texts per language, translation spot-checks.

B.2 Handbook go-live (PR #516 / #394, [#530-E])

  1. Create the HANDBOOK_SYNC_TOKEN PAT (write access to immersive-app/handbook) and add it as a secret on this repo — until it exists sync-handbook.yml no-ops with a clear log.
  2. Trigger/land a docs change → the sync workflow commits into the handbook repo.
  3. Cloudflare Pages builds the Jekyll site.
  4. Cloudflare Access app in front of it: opening the handbook URL prompts for email OTP.

Verify: this very page appears in the handbook under Engineering → Technical after the first sync.

B.3 Cloudflare leftovers (#505)

Remaining unchecked items on the issue:

  1. Enable the WAF managed ruleset + a login/API rate-limit rule at the edge (free tier), Cloudflare dashboard → Security.
  2. Mail delivery confirmation post-flip: send + receive on the domain mailbox and one MailerSend transactional (password reset) — MX/SPF/DKIM must be untouched (grey-cloud).
  3. Confirm real client IPs reach Rack::Attack/Ahoy through the proxy (see 13.2), then tick the issue.
  4. Update docs/technical/production/dns.md if any of the above changed records.

Verify: 13.2’s throttle log shows an end-user IP; a WAF-blocked test request (e.g. an obvious SQLi probe via curl) returns a Cloudflare challenge/block page, and Sentry stays quiet.

B.4 ElevenLabs voice pass (#415, [#530-D])

The ~30-minute voice-selection listening pass in the ElevenLabs studio: choose 1 male + 1 female voice per language — Catalan and pt-PT are make-or-break. While in the account, confirm it is genuinely on a paid plan (the dev key still reports free-tier).

Verify: chosen voice IDs recorded in the voice configuration; a reader playback in ca and pt uses them.

B.5 iOS shell first-run on hardware (PRs #502/#520 / #466, [#530-C])

Not automatable: the Section 9 run on a Mac with full Xcode is itself the pending action until the beta ships. Follow-ups tracked in #521 (tab-title localization, canonical prod host).

Verify: Section 9 passes end-to-end in the simulator.

This site uses Just the Docs, a documentation theme for Jekyll.