Documents Feature

Documents have two modes: being read or written.

The Coach is available in either mode. The coach is an AI assistant that offers explanations and help in composing text and ideas for writing. It runs through Resources::Documents::CoachService, which scopes the request with Chat::ContextBuilder (user, surface, document, language pair) and asks the LLM (Resources::Documents::CoachPrompt) for feedback in the user’s fluent language; the reply renders in the coach-result panel on the document form.

Documents are a full page view, and there is an icon and link at the top of the page. When the Coach has something to say, this icon changes colour and flashes. Clicking the icon takes us to the Coach chat window. In this window, the relevant part of the document is extracted for better viewing.

It is also possible to choose a 2 column view, where the chat is 1/3 of the page and the document 2/3.

Writing

The user writes and submits the document, the Language Coach then assesses the work and gives feedback. See the coach.md document for a description on the coach.

The Correct action on the edit form submits the document’s text as a WritingSubmission, corrected asynchronously by Writing::CorrectionJob → Writing::CorrectionService (schema-constrained LLM call): structured corrections with explanations in the fluent language, register alternatives (natural/simpler/formal/casual), and a CEFR estimate. From the submission page the corrected text can be applied back to the document — this records the replaced state as an ai_correction revision and re-runs segmentation (markdown formatting is dropped, since the correction is plain text).

The writing assistant panel (#427)

The write view carries a three-tab assistant (design 2h/2i/2j/2m): a sticky right panel on desktop, a bottom sheet behind a toggle on mobile, driven by writing_assistant_controller.js on the editor wrapper (the panel sits outside the document form because it carries its own .docx export form).

• Edits hosts the correction flow: the newest submission’s corrections as compact cards, the writer’s CEFR level, and the Correct action; the full report stays on the submission page.
• Rewrite is per-selection: the writer selects a passage in the markdown textarea, and documents#rewrite (synchronous schema-constrained call, RewritePrompt, faked in dev) returns a version one CEFR step above the writer’s level (WriterLevel, read from the newest corrected submission, default A2). The suggestion lands under the paragraph, in the document — a card positioned over the editor paper by a textarea mirror measurement — with Replace (splices the selection via setRangeText) and Dismiss.
• Tone takes a free-text instruction (“a formal email to my landlord about the broken heater”) and rewrites the whole draft inside the writer’s level range (TonePrompt). The result offers Apply to document (fills the editor; Save persists and records the revision) and Export .docx — generated by DocxExporter, a minimal rubyzip-based writer (title + plain paragraphs; no docx gem). Tone drafts are ephemeral until applied.

Reading

The user can upload and read any document. The document is split into document_words and document_sentences when saved.

There is the possibility to view the document by words or sentences. Clicking on a sentence or word will reveal the translation in the user’s fluent language. Clicking a second time will play spoken audio. Clicking again or clicking on another sentence will pause the running audio.

These interactions feed the user’s vocabulary list: translating a document word or sentence records a VocabularyEntry via Vocabulary::Collector (translate event, with the translation and source token), and word/sentence reviews recorded through Reviews::FindOrCreateService count as review — or miss, for again/hard ease. Entries are unique per user/language/term (words case-insensitive) and carry counters plus last_event_at for the future Profile vocabulary and progress surfaces. Collection is best-effort and never breaks the triggering action.

The reader toolbar has a voice picker: the chosen text-to-speech voice is saved implicitly as the user’s default for the document’s language (a Sticky Setting, document_voices), validated against VoiceCatalog, and preferred by audio synthesis. Resources::Documents::ReadAloudPlanner and DictationPlanner build “Read to me” and dictation sequences (voice, slow rate, sentence pauses, key-word repeats, typing pauses) served as JSON from /documents/:id/read_aloud_plan and /documents/:id/dictation_plan; the dictation player is a later slice.

Follow-along playback (#415, design 2c)

“Read to me” in the control sheet starts synchronized listen-while-reading, driven by reader_playback_controller.js consuming the read-aloud plan. The sentence being read is highlighted in the text (.reader-sentence--playing, the selection tint on every span carrying that data-document-sentence-id — the sentence wrapper in sentence view, each word span in word view) and scrolled into view as playback advances. Tapping another sentence while playing jumps there; while paused, taps keep their normal reading behavior (translation popover, aid layer). The transport — pause/resume, elapsed/total with a progress track, and a 1×/0.7× speed toggle (ReadAloudPlanner::SLOW_RATE, applied as playbackRate) — lives inside the existing bottom aid bar, which the “Last word” aid display yields to during playback. The whole document is read in a single voice (#586): the plan adopts the sticky voice — picking and persisting a random catalog voice on first use — and only embeds a step’s audio_url when the attached audio was synthesized with that voice (recorded as voice_id in the blob’s metadata); other steps, including pre-#586 audio with no recorded voice, are generated on demand through POST /api/resources/audio with the plan voice passed along, and AttachAudioService replaces an attachment whose stored voice differs from the requested one. Sentences are never silently skipped: a failed step is retried a couple of times, a 429 from the synthesis throttle is waited out via Retry-After, and a buffered clip that refuses to start (typically an expired signed URL on a long document) is re-resolved through the same generate-on-miss path. Between sentences the planner inserts only a short constant pause (PAUSE_MS), so playback reads as continuous text. Preloading follows #374’s one-ahead strategy: while sentence N plays, sentence N+1’s audio is prepared and buffered (preload="auto"), bounding synthesis to one in-flight request. Sentence-level sync is the free tier per the #418 decision; word-level follow (ElevenLabs/Polly speech-mark timing) is the premium tier and not built yet.

Both views render server-side through Resources::Documents::ReaderRenderer. For documents imported from markdown, the renderer preserves allowed formatting (headings, bold/italic, lists, blockquotes, code, tables) while wrapping each stored token in the interactive span structure, tagged with data-document-word-id and data-document-sentence-id. The markdown text is re-aligned against the stored document_words/document_sentences streams; if they cannot be aligned (or the document has no markdown, e.g. pasted text, plain files, PDFs, images), it falls back to the unformatted token stream. Raw HTML in content is always escaped.

Reading aids (the EuRom5 comprehension-aid layer, #411)

The reader carries the inline aid layer defined in docs/features/bridge_method.md. Aids are data, not ephemeral output: document_aid_sets is the generation unit — one row per (document, known-language-set), where the set key is the learner’s known Bridge languages in canonical continuum order — and document_aids holds the individual aids, each anchored to exactly one document_word or document_sentence (never a line number; decision on #389). Aid types: phantom (inference nudge with calque/sense-gloss behind a second step), skip, listen, morphology, already_seen, and grammar_ref (payload carries a phenomenon_key resolving to a reading_phenomena row - the grammar-of-reading structure from #412, see docs/features/grammar_of_reading.md; not LLM-generated until phenomenon content is authored).

Generation is lazy: opening the reader calls AidGenerationService.ensure_for, which creates the set for the reader’s current known-language-set and enqueues AidGenerationJob (schema-constrained LLM call via AidGenerationPrompt, faked in dev by FakeAI). The service resolves the LLM’s word/sentence indexes against the stored token streams and drops anything whose token doesn’t match, so misalignment can lose an aid but never mis-anchor one. Regeneration replaces source: llm rows and preserves source: human rows — the human update path is the Document Aids ActiveAdmin resource, whose edits are marked source: human.

Rendering: AidFilter applies the scaffolding-fade rule — an aid shows while the learner’s user_languages level in the document’s language is at or below the aid’s fade_level; native readers get none — and tiers the survivors (light = would still show two levels up). ReaderRenderer marks aided spans with the single .reader-aid underline plus data-aid-type/data-aid-tier/data-aid-payload. The popover reveals the aid content: phantom words get progressive disclosure (nudge first, glosses behind the reveal step), morphology shows its decomposition, already-seen links jump to the earlier occurrence, and word aids with equivalents render the six-language column in continuum order PT–ES–CA–IT–FR–EN with the learner’s languages in full ink. The aid bar shows the aid’s bridge cue as a chip. The control sheet’s Aids row (Full/Light) is a sticky setting (reader_aid_density) toggled client-side via the tier annotations.

Completing a phantom word’s reveal creates a document.word.inference.v1 card — cloze-by-inference: the sentence with the word masked, answered by the word and its gloss — deduped per document word, linked to the DocumentWord through card_sources (inference_context), in an auto-created per-language “From reading” deck. These are the first tenants of the card template layer (template_key/template_version/prompt_payload/answer_payload on cards; see docs/features/review_card_spec.md).

The Document Library (#429)

Curated, licence-clean texts provided by Immersive — never LLM-generated, never EuRom5’s own texts (#407). A library document is a document owned by the Immersive user (User::IMMERSIVE_USER_ID) with published: true (Document.library); curation happens through the Documents ActiveAdmin resource, where published gates visibility. #416 (graded reading library) is the content pipeline that ships onto this plumbing.

Any signed-in learner can read a library document (DocumentPolicy#show?), including the aid layer — for library documents the reader who first triggers generation for a known-language-set supplies that set’s gloss language, since the Immersive owner’s fluent language is meaningless. Switching to write is copy-on-write (Resources::Documents::CopyOnWriteService): the edit entry creates a copy in the learner’s own files (provenance via documents.copied_from_id), re-runs segmentation and title gloss for the new owner, and lands them in the editor with a notice making the moment legible. One copy per (user, library document): a second Write tap opens the existing copy. The library original is never editable by learners (update? stays owner-only).

The index shows a Library section at the root (design 2n: title, gloss, level, word count, Browse all → /documents/library), and the empty state gains the second door, “Browse the library” (design 2o).

Seeding (#489): the first library texts are vendored in db/seeds/library_documents/ — one YAML file per Bridge language, three verbatim Wikipedia lead-section excerpts each (CC BY-SA 4.0; provenance per entry and in SOURCES.md there). bin/rails immersive:seed_library_documents runs Resources::Documents::LibrarySeeder: idempotent, keyed by (Immersive user, language, title), runs the normal ProcessJob pipeline on new or changed text, and sets level from the seed data’s #416 word-count tranche (A 100–200 → beginner, B 200–300 → intermediate) because EvaluateLevelService is still a stub. Adding a text = adding a YAML entry and re-running the task.

Edit history

Every content or title change snapshots the state it replaces into document_revisions (Resources::Documents::RevisionRecorder; edits snapshot in the controller, imports in ImportService, and ai_correction is reserved for the writing-correction work). The History link on the edit form opens /documents/:id/revisions: a list of past versions with a compare view (line diff against the current version) and restore. Restoring records the pre-restore state first, writes the snapshot back, and re-runs segmentation.

Documents and folders

When accessing the Documents link from the menu, the user is taken to the documents index page. If there are no existing documents or folders, and empty state dialog is presented with the ability to create either a folder or a document. Otherwise there are toolbar links (using the same CSS and html as decks#index) to create or folder documents and folders.

There is a no root folder icon, but there is the concept of a root. Documents belong to folders and folders can nest to any level.