LLM Prompt Architecture (RubyLLM)

This document describes how LLM calls are structured in the application.

The single approach

All LLM access goes through RubyLLM (#499). There is no other LLM client:

• BasePrompt (app/prompts/) is the standard entry point: YAML prompt files in config/prompts/, Schematist::Schema classes in app/schemas/, value objects in app/values/.
• BaseLLMService (app/services/base_llm_service.rb) is a thin RubyLLM.chat wrapper used by three older services; folding them into BasePrompt classes is deferred (see #499).
• Embeddings use RubyLLM.embed directly (Sentences::EmbedService).
• FakeAI (docs/technical/fake_ai.md) fakes all three entry points in development, and Mocha stubs cover them in tests; LIVE_MODE=true switches either to the real APIs.

Goals

• Single-responsibility classes
• Clear separation between:
  • Prompt definition + LLM concerns
  • Domain orchestration and DB updates
• Local prompt files so the full context is visible in-editor (e.g. Cursor)
• Easy extension to tools, RAG, streaming, etc.
• Resilient error handling with automatic retries for transient failures

Layers

1. BasePrompt (app/prompts/base_prompt.rb)

BasePrompt centralizes all cross-cutting concerns:

• Loads prompt text from YAML (config/prompts/...)
• Interpolates variables (e.g. ,)
• Builds chat messages (system and user)
• Selects the model (with optional override)
• Attaches a RubyLLM Schema (optional)
• Converts the response into a value object

Subclasses override small, focused methods:

• prompt_file_path – where the YAML lives
• schema_class – Schematist::Schema subclass (optional)
• value_class – Value object class (optional)
• model_name – default model for this prompt

2. Prompt YAML files (config/prompts/...)

Each prompt lives in its own YAML file, e.g.:

• config/prompts/resources/verbs/common_classifier_prompt.yml
• config/prompts/resources/verbs/verbals_generator_prompt.yml
• config/prompts/resources/verbs/translations_generator_prompt.yml
• config/prompts/resources/verbs/conjugations_generator_prompt.yml

Structure:

system: |-
  System message with .
user: |-
  User message with .

The YAML is the canonical source of prompt text:

• Easy to edit
• Fully visible to the editor (Cursor)
• Simple to version in Git

3. Schemas (app/schemas/…)

We use Schematist::Schema (the schematist gem, formerly ruby_llm-schema) to describe the expected JSON shape from the LLM:

• Resources::Verbs::CommonClassifierSchema
• Resources::Verbs::VerbalsSchema
• Resources::Verbs::TranslationsSchema
• Resources::Verbs::ConjugationsGeneratorSchema

These schemas are passed to RubyLLM, so the model is instructed to output well-structured JSON.

4. Value objects (app/values/…)

Each schema has a corresponding value object:

• CommonClassifierValue
• VerbalsValue
• TranslationsValue
• ConjugationsGeneratorValue

They encapsulate:

• Small accessors (common, confidence, reasons, etc.)
• Error helpers (error?, valid?)
• Normalization (to_h for verbals)

The BasePrompt converts response.content into the appropriate value object.

5. Prompt subclasses (app/prompts/resources/verbs/…)

Examples:

• Resources::Verbs::CommonClassifierPrompt
• Resources::Verbs::VerbalsGeneratorPrompt
• Resources::Verbs::TranslationsGeneratorPrompt
• Resources::Verbs::ConjugationsGeneratorPrompt

Each prompt subclass:

• Points to its YAML file
• Declares its schema + value class
• Declares the model to use

They do not contain any domain logic (no DB updates, no branching for “common vs uncommon”). They only describe how to talk to the LLM for a specific task.

6. Service Classes (app/services/resources/verbs/…)

Each prompt has a corresponding service class that handles the business logic:

• Resources::Verbs::CommonClassifierService – calls CommonClassifierPrompt and updates verb_lemma.common
• Resources::Verbs::VerbalsGeneratorService – calls VerbalsGeneratorPrompt and creates Verbal records
• Resources::Verbs::TranslationsGeneratorService – calls TranslationsGeneratorPrompt and creates/updates VerbLemmaTranslation records
• Resources::Verbs::ConjugationsGeneratorService – calls ConjugationsGeneratorPrompt and creates Conjugation records, enqueues audio job

Services handle:

• Validation of input (verb_lemma must be persisted, have language, etc.)
• Calling the appropriate prompt
• Persisting results to the database
• Returning result hashes with :status (:success or :error) and relevant data

7. Jobs (app/jobs/resources/verbs/…)

Each verb validation service has a corresponding background job (queue: llm):

• Resources::Verbs::ValidateCommonJob
• Resources::Verbs::ValidateVerbalsJob
• Resources::Verbs::ValidateTranslationsJob
• Resources::Verbs::ValidateConjugationsJob

Each job loads the VerbLemma, calls its service, and sets the matching *_validated flag on success. Resources::VerbsController#show enqueues them lazily for unvalidated verbs; the admin VerbLemmas validate action runs the services synchronously. See docs/features/verbs/verb_validator.md.

The jobs do not contain any LLM call logic or persistence logic – they only coordinate the services.

8. Streaming

In this design:

Schema-based prompts are non-streaming for simplicity and reliability.

The BasePrompt explicitly forbids streaming? == true when a schema_class is provided.

In a future iteration, streaming can be enabled for:

• Prompts with no schema
• Chat-style responses where streaming is useful (e.g. long explanations in the UI)

9. Why this design works well

Single responsibility:

• Prompt classes deal only with LLM communication.
• Service classes deal with business logic and persistence.
• Orchestrators deal only with coordination.

Testable:

• Prompts can be unit-tested by stubbing RubyLLM.
• Services can be tested by stubbing prompts.
• Orchestrators can be tested by stubbing services.

Editor-friendly:

• Prompt text lives in YAML files, visible to the IDE.

Extensible:

• New prompts are just new subclasses + YAML files + schemas/values.
• New services follow the same pattern.
• Tools and RAG can be added by extending BasePrompt.