# airobal/barprep-by-shep Wiki
Version: 1
## Overview
### Introduction & Product Overview
Relevant source files
The following files were used as context for generating this wiki page:
- [README.md](https://github.com/airobal/barprep-by-shep/blob/main/README.md)
- [PRODUCT\_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
# Introduction & Product Overview
Bar Prep by SHEP is a specialized, standalone bar examination preparation application derived from the SHEP (ShepardShallKnowIt) core product. It is designed to provide structured legal reasoning practice for two primary examination formats: the **NextGen UBE** (launching July 2026) and the traditional **Multistate Essay Examination (MEE)**. The product is positioned as a high-fidelity practice tool that replicates the official exam environment and uses licensed NCBE content.
The application operates as a separate product with its own authentication scope and database, though it maintains a relationship with the SHEP core system for unified user management. It is built using a modern technology stack including SvelteKit 2, Svelte 5, and Tailwind CSS 4, utilizing SHEP's existing design system to maintain high craft standards and professional aesthetics.
Sources: [HANDOFF.md:7-13](), [PRODUCT\_SPEC.md:7-13](), [README.md:5-10]()
## Core Exam Domains
The system is architected around two distinct exam domains, each with specific requirements for materials, timing, and evaluation.
### NextGen UBE
The NextGen bar exam focuses on skills-based testing. The application supports three primary task types within this domain:
* **SPT (Standard Performance Task):** Involves reading provided materials and drafting legal documents like memos or briefs.
* **LRPT (Legal Research Performance Task):** Focused on legal research and drafting.
* **IQS (Integrated Question Set):** Features a progressive disclosure flow for multi-step questions.
Sources: [README.md:37-41](), [PRODUCT\_SPEC.md:31-41]()
### Multistate Essay Examination (MEE)
The MEE domain focuses on traditional legal recall across 13 subject areas. Unlike NextGen, no research materials are provided during the exam, requiring pure recall and application of legal principles using the IRAC (Issue, Rule, Analysis, Conclusion) structure.
Sources: [README.md:43-46](), [PRODUCT\_SPEC.md:43-51]()
### Comparison Table
| Aspect | NextGen UBE | MEE |
| :--- | :--- | :--- |
| **Primary Focus** | Skills (Reading, Research, App) | Pure Recall & IRAC Application |
| **Materials** | Legal sources library provided | None provided |
| **Timing** | 24–60 min per task (variable) | 30 min per essay (fixed) |
| **Task Types** | SPT, LRPT, IQS | 6 Essays |
| **Rubric** | 4-Dimension Competency | Issue-based scoring |
Sources: [PRODUCT\_SPEC.md:21-29](), [README.md:35-46]()
## System Architecture
### Frontend Technology Stack
The application is built using a monorepo structure with npm workspaces.
* **Framework:** SvelteKit 2 with Svelte 5 (Runes).
* **Styling:** Tailwind CSS 4 using SHEP's design tokens (OKLCH color space, typography, and motion).
* **State Management:** Module-level `$state` for non-user-specific global state (theme, device) and factory functions (`createPracticeState`) for per-instance practice sessions.
Sources: [CLAUDE.md:14-25](), [HANDOFF.md:65-71]()
### Dual-Database Architecture
The system employs a specific two-database strategy to integrate with the SHEP core ecosystem while maintaining data isolation for bar-prep specific features.
```mermaid
graph TD
subgraph BarPrep_Environment
App[SvelteKit App] --> BP_DB[(Barprep DB)]
App --> AUTH_DB[(SHEP Core Auth DB)]
end
BP_DB -- "Foreign Key (Unenforced)" --> AUTH_DB
subgraph BP_DB_Tables
U_BP[Users Shadow Table]
SESS[Practice Sessions]
DRAFT[Drafts]
SUB[Submissions]
end
subgraph AUTH_DB_Tables
U_CORE[Users Table]
end
U_BP -- "auth_user_id" --> U_CORE
```
*The diagram illustrates the dual-database architecture where Barprep maintains its own exam data while syncing users to the SHEP core database.*
* **DATABASE_URL:** Stores Barprep-specific data including shadow users, sessions, tasks, drafts, and submissions.
* **AUTH_DATABASE_URL:** Points to the SHEP core database. Auth.js uses this for user registration so Barprep users appear in the core SHEP user count.
Sources: [CLAUDE.md:44-59](), [apps/web-svelte/database/migrations/001_initial_schema.sql:43-46]()
## Data Models & Schema
The database schema is designed to handle both exam types through polymorphic references and specific specialized tables.
### Shared Infrastructure
Core tables are shared across both exam types to manage user progress and persistence.
* **Users:** Links to SHEP core via `auth_user_id` and tracks `exam_type` (NextGen or MEE).
* **Practice Sessions:** Tracks session status (`IN_PROGRESS`, `COMPLETED`, `ABANDONED`), timing, and results.
* **Drafts:** Manages auto-saves for active writing sessions.
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:41-94]()
### Domain-Specific Tables
| Domain | Key Tables | Description |
| :--- | :--- | :--- |
| **NextGen** | `nextgen_tasks`, `nextgen_legal_materials` | Stores scenarios, library materials, and role prompts. |
| **MEE** | `mee_essays`, `mee_model_answers` | Stores essay prompts and licensed model answers for comparison. |
| **Evaluation** | `nextgen_submissions`, `mee_submissions` | Stores user responses and rubric-based scores. |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:103-176](), [PRODUCT\_SPEC.md:76-90]()
## Product Features & UX
### Practice Environment ("The Chambers")
For NextGen tasks, the UI replicates the official NCBE interface, featuring:
* **Split-pane Layout:** A draggable divider separating materials from the writing area.
* **Materials Panel:** Tabbed navigation for statutes, cases, and client documents.
* **Timer:** A "gentle horizon" timer with visibility toggles and threshold warnings.
Sources: [HANDOFF.md:31-41](), [PRODUCT\_SPEC.md:64-74]()
### Evaluation System
The application provides structured feedback based on domain-specific rubrics:
* **NextGen Rubric:** Evaluates 4 dimensions: Issue ID, Rule Statement, Rule Application, and Communication.
* **MEE Rubric:** Uses a traditional issue-based breakdown and allows side-by-side comparison with licensed model answers.
Sources: [README.md:41-46](), [PRODUCT\_SPEC.md:67-73]()
## Conclusion
Bar Prep by SHEP provides a high-fidelity, skills-focused practice environment for modern bar exams. By leveraging the SHEP core design system and a robust dual-database architecture, the project delivers a professional-grade tool capable of handling complex NextGen performance tasks and traditional MEE essays, while remaining integrated into the broader SHEP user ecosystem.
Sources: [HANDOFF.md:7-13](), [PRODUCT\_SPEC.md:7-15]()
### Getting Started & Setup
Relevant source files
The following files were used as context for generating this wiki page:
- [README.md](https://github.com/airobal/barprep-by-shep/blob/main/README.md)
- [package.json](https://github.com/airobal/barprep-by-shep/blob/main/package.json)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [apps/web-svelte/database/MIGRATIONS.md](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/MIGRATIONS.md)
- [apps/web-svelte/package.json](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/package.json)
# Getting Started & Setup
This guide provides the necessary technical information to initialize, configure, and run the Bar Prep by SHEP application. Bar Prep by SHEP is a standalone legal reasoning practice tool designed for the NextGen Bar Exam (July 2026+) and the Multistate Essay Examination (MEE). It leverages a SvelteKit 5 architecture and a dual-database Neon PostgreSQL backend to provide a realistic exam simulation environment.
Sources: [README.md:1-7](), [HANDOFF.md:12-16](), [CLAUDE.md:7-12]()
## Prerequisites & Environment
The project is structured as a monorepo using **npm workspaces**. The primary application logic resides in `apps/web-svelte/`, while licensed content ingestion logic is housed in `packages/content-pipeline/`.
### System Requirements
| Requirement | Value | Source |
| :--- | :--- | :--- |
| **Node.js** | >= 20.0.0 | [package.json:23]() |
| **Package Manager** | npm | [package.json:5-13]() |
| **Database** | Neon PostgreSQL | [PRODUCT_SPEC.md:131]() |
| **OS** | Linux/macOS/Windows | [CLAUDE.md:16-25]() |
### Core Tech Stack
* **Frontend**: SvelteKit 2 + Svelte 5 (Runes)
* **Styling**: Tailwind CSS 4 (using `@tailwindcss/vite`)
* **Authentication**: Auth.js (`@auth/sveltekit`)
* **Database**: Postgres (using `postgres` and `@auth/pg-adapter` drivers)
* **Validation**: Zod
Sources: [CLAUDE.md:27-32](), [apps/web-svelte/package.json:42-61]()
---
## Initial Installation
To set up the development environment, follow these steps from the repository root:
1. **Install Dependencies**:
```bash
npm install
```
Sources: [README.md:11-12]()
2. **Environment Configuration**:
Copy the example environment file and populate it with your credentials:
```bash
cp .env.example .env
```
Sources: [README.md:14-15]()
### Required Environment Variables
The application utilizes a dual-database architecture. `DATABASE_URL` is used for application-specific data (sessions, tasks, drafts), while `AUTH_DATABASE_URL` connects to the SHEP core database for unified user management.
| Variable | Description |
| :--- | :--- |
| `DATABASE_URL` | Primary Barprep database URL (Neon) |
| `AUTH_DATABASE_URL` | SHEP core database URL for Auth.js |
| `BARPREP_DATABASE_URL` | Alias used by migration scripts |
| `BARPREP_SCRATCH_DATABASE_URL` | Used for generating schema snapshots |
Sources: [CLAUDE.md:46-59](), [apps/web-svelte/database/MIGRATIONS.md:50-55]()
---
## Database & Migrations
The database lifecycle is managed through SQL migration files located in `apps/web-svelte/database/migrations/`.
### Migration Workflow
The project follows a "forward-only" migration history with a canonical schema snapshot.
```mermaid
flowchart TD
A[Create SQL Migration in /migrations] --> B[Run npm run migrate]
B --> C{Apply to DATABASE_URL}
C --> D[Run npm run migrate:snapshot]
D --> E[Update apps/web-svelte/database/schema.sql]
E --> F[Commit SQL + schema.sql]
```
This diagram illustrates the mandatory workflow for ensuring the local schema remains in sync with the canonical snapshot.
### Management Commands
These commands can be run from the root or within the `apps/web-svelte/` workspace:
* **Apply Migrations**: `npm run migrate` (Applies pending `.sql` files to `BARPREP_DATABASE_URL`).
* **Generate Snapshot**: `npm run migrate:snapshot` (Replays history against a scratch DB to update `schema.sql`).
Sources: [README.md:17-27](), [apps/web-svelte/database/MIGRATIONS.md:5-13](), [apps/web-svelte/database/MIGRATIONS.md:40-45]()
---
## Development & Testing
Once the environment and database are configured, the application can be started for local development.
### Running the Application
```bash
npm run dev
```
By default, the Vite development server runs on **port 5175**.
Sources: [CLAUDE.md:18]()
### Command Reference
| Task | Command | Description |
| :--- | :--- | :--- |
| **Build** | `npm run build` | Produces the production build |
| **Type Check** | `npm run check` | Runs svelte-check and sync |
| **Linting** | `npm run lint` | Runs ESLint |
| **Unit Tests** | `npm run test` | Executes Vitest suite |
| **E2E Tests** | `npm run test:e2e` | Runs Playwright tests from `apps/web-svelte` |
Sources: [CLAUDE.md:18-24](), [apps/web-svelte/package.json:6-15]()
---
## Project Architecture Overview
The system architecture emphasizes domain-driven organization. Logic is separated by exam type (`nextgen` vs `mee`) rather than technical type (e.g., components vs services).
### Application Layout
```mermaid
graph TD
Root[barprep-by-shep] --> Apps[apps/web-svelte]
Root --> Packages[packages/content-pipeline]
Apps --> Lib[src/lib]
Apps --> Routes[src/routes]
Lib --> Shared[shared - Cross-exam types]
Lib --> NextGen[exam-types/nextgen]
Lib --> MEE[exam-types/mee]
Routes --> RNextGen[/nextgen]
Routes --> RMEE[/mee]
```
This structure ensures that code specific to the NextGen bar exam remains isolated from MEE logic, while shared utilities (like Authentication and User types) are accessible to both.
Sources: [CLAUDE.md:34-44](), [README.md:31-45]()
## Conclusion
Setting up the Bar Prep by SHEP project involves configuring a SvelteKit workspace and a dual-database PostgreSQL backend. By adhering to the migration workflow and using the provided npm scripts, developers can maintain a stable environment that mirrors the SHEP core design system while developing isolated bar exam preparation features.
Sources: [HANDOFF.md:12-16](), [CLAUDE.md:46-59]()
### Exam Types: NextGen vs MEE
Relevant source files
The following files were used as context for generating this wiki page:
- [PRODUCT\_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [README.md](https://github.com/airobal/barprep-by-shep/blob/main/README.md)
- [apps/web-svelte/src/lib/exam-types/mee/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/mee/types.ts)
- [apps/web-svelte/src/lib/exam-types/nextgen/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/nextgen/types.ts)
- [apps/web-svelte/src/lib/exam-types/mee/constants.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/mee/constants.ts)
- [apps/web-svelte/src/lib/exam-types/nextgen/constants.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/nextgen/constants.ts)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
# Exam Types: NextGen vs MEE
## Introduction
Bar Prep by SHEP is designed as a dual-track preparation platform catering to two distinct bar examination formats: the **NextGen UBE** (launching July 2026) and the traditional **Multistate Essay Examination (MEE)**. While sharing a unified core for user authentication, session tracking, and draft management, the application bifurcates its internal logic, UI components, and data structures to accommodate the unique pedagogical and structural requirements of each exam type.
Users select their specific exam path during onboarding, which determines the available practice modules, scoring rubrics, and the interface provided within the "The Chambers" writing environment. NextGen focuses on skills-based tasks like research and application using provided legal libraries, whereas MEE emphasizes pure recall and the IRAC (Issue, Rule, Analysis, Conclusion) structural framework.
Sources: [PRODUCT\_SPEC.md:7-13](), [README.md:5-10](), [apps/web-svelte/database/migrations/001\_initial_schema.sql:35-46]()
## Architectural Comparison
The project maintains a clear separation of concerns between exam types while utilizing shared infrastructure for persistence and session state.
### High-Level Domain Model
The following diagram illustrates how the shared user and session entities relate to the domain-specific content of NextGen and MEE.
```mermaid
erDiagram
USER ||--o{ PRACTICE_SESSION : "starts"
USER {
uuid id
exam_type current_path
subscription_status status
}
PRACTICE_SESSION ||--o| NEXTGEN_TASK : "references"
PRACTICE_SESSION ||--o| MEE_ESSAY : "references"
PRACTICE_SESSION {
uuid id
exam_type type
int elapsed_seconds
status session_status
}
NEXTGEN_TASK ||--o{ LEGAL_MATERIAL : "contains"
MEE_ESSAY ||--o| MODEL_ANSWER : "has"
```
The system uses polymorphic references in the `practice_sessions` and `drafts` tables, enforced by SQL `CHECK` constraints to ensure a session points to either a NextGen task or an MEE essay, never both.
Sources: [apps/web-svelte/database/migrations/001\_initial_schema.sql:48-93](), [PRODUCT\_SPEC.md:71-92]()
### Format and Skills Matrix
| Aspect | MEE | NextGen |
| :--- | :--- | :--- |
| **Primary Focus** | Issue spotting & IRAC recall | Reading, research, and application |
| **Material Access** | None (Closed book/Pure recall) | Integrated Legal Sources Library |
| **Task Types** | 6 Essays | SPT, LRPT, and IQS |
| **Timing** | Fixed (30 min per essay) | Variable (24-60 min per task) |
| **Scoring Logic** | Issue-based breakdown | 4-Dimension Competency Rubric |
Sources: [PRODUCT\_SPEC.md:21-33](), [README.md:38-51](), [apps/web-svelte/src/lib/exam-types/nextgen/constants.ts:68-72]()
---
## NextGen Bar Exam Module
The NextGen module replicates the NCBE skills-based interface, focusing on legal research and document drafting.
### Task Types and Metadata
NextGen content is categorized into three primary functional types:
1. **SPT (Standard Performance Task):** Involves reading materials and completing a complex writing assignment (e.g., a memo or brief) based on a supervisor's instructions.
2. **LRPT (Legal Research Performance Task):** Focuses on researching specific issues using provided sources.
3. **IQS (Integrated Question Set):** Features progressive disclosure where information and questions are revealed in steps.
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/types.ts:7-22](), [apps/web-svelte/src/lib/exam-types/nextgen/constants.ts:11-23]()
### NextGen Data Structures
```typescript
export interface NextGenTask {
id: string;
title: string;
taskType: 'SPT' | 'LRPT' | 'IQS';
skillsGroup: 'A' | 'B' | 'C' | 'D';
foundationalConcepts: string[];
expectedTimeMinutes: number;
rolePrompt: string;
supervisorMemo: SupervisorMemo | null;
disclosureSteps: DisclosureStep[];
}
```
NextGen tasks are supported by a `nextgen_legal_materials` table containing various types of documentation such as `STATUTE`, `CASE_EXCERPT`, `REGULATION`, and `CLIENT_DOCUMENT`.
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/types.ts:24-37](), [apps/web-svelte/database/migrations/001\_initial_schema.sql:113-132]()
### Scoring: 4-Dimension Rubric
NextGen performance is evaluated across four competency dimensions rather than specific legal issues:
* **Issue Identification** (20% weight)
* **Rule Statement** (25% weight)
* **Rule Application** (35% weight)
* **Communication** (20% weight)
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/constants.ts:31-47](), [apps/web-svelte/src/lib/exam-types/nextgen/types.ts:65-74]()
---
## MEE Module
The MEE module provides a streamlined interface for traditional essay writing, emphasizing mastery of core legal subjects.
### Subject Coverage
The MEE implementation tracks coverage across 13 specific subjects, divided into core and elective areas.
```mermaid
flowchart TD
subgraph Core_Subjects
CP[Civil Procedure]
CL[Constitutional Law]
CT[Contracts]
CR[Criminal Law]
EV[Evidence]
RP[Real Property]
TO[Torts]
end
subgraph Elective_Subjects
BA[Business Associations]
COL[Conflict of Laws]
FL[Family Law]
TE[Trusts & Estates]
ST[Secured Transactions]
UCC[UCC Article 9]
end
```
Sources: [apps/web-svelte/src/lib/exam-types/mee/types.ts:7-21](), [apps/web-svelte/src/lib/exam-types/mee/constants.ts:31-39]()
### MEE Essay Data Model
The MEE data model focuses on the fact pattern, the "call of the question," and comparison against licensed model answers.
| Field | Type | Description |
| :--- | :--- | :--- |
| `fact_pattern` | Text | The scenario presented to the candidate. |
| `call_of_the_question` | Text | The specific legal instructions/questions to answer. |
| `subject` | Enum | The primary legal area (e.g., TORTS). |
| `difficulty` | Enum | Foundational, Intermediate, or Advanced. |
| `model_answer` | Relation | Links to licensed NCBE or SHEP model answers. |
Sources: [apps/web-svelte/database/migrations/001\_initial_schema.sql:151-176](), [apps/web-svelte/src/lib/exam-types/mee/types.ts:27-39]()
### IRAC Template and Scoring
To assist with structure, the MEE module provides an optional IRAC template. Scoring is issue-based, awarding points based on the quality of the rule statement and application for each sub-issue identified in the model answer.
Sources: [apps/web-svelte/src/lib/exam-types/mee/constants.ts:57-73](), [apps/web-svelte/src/lib/exam-types/mee/types.ts:51-64]()
## Summary
The distinction between NextGen and MEE exam types in Bar Prep by SHEP is fundamental to the application's architecture. NextGen is characterized by complex data relationships involving legal material libraries and multi-step disclosure logic, whereas MEE follows a more traditional, subject-centric structure. This separation allows the platform to support the transition from the legacy UBE format to the new NextGen standard within a single unified codebase.
Sources: [PRODUCT\_SPEC.md:120-135](), [README.md:14-25]()
## System Architecture
### Dual-Database Architecture
Relevant source files
The following files were used as context for generating this wiki page:
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
- [apps/web-svelte/database/migrations/005_stripe_fields.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/005_stripe_fields.sql)
# Dual-Database Architecture
The Barprep application utilizes a Dual-Database Architecture to manage data isolation while maintaining integration with the core SHEP (ShepardShallKnowIt) ecosystem. This architecture separates application-specific exam data from centralized user authentication and identity management. By splitting concerns between two Neon PostgreSQL databases, the system ensures that Barprep users are accounted for in the primary SHEP user metrics while keeping sensitive exam progress and licensed content isolated.
The primary objective of this structure is to allow Barprep to operate as a standalone product while sharing a design system and authentication scope with SHEP core. All application-specific queries are directed to the Barprep database, whereas authentication-related persistence and registration data flow to the SHEP core database.
Sources: [CLAUDE.md:65-75](), [PRODUCT_SPEC.md:15-25](), [HANDOFF.md:10-15]()
## Architecture Overview
The system architecture consists of two distinct databases within the same Neon project, each serving a specific role in the application lifecycle.
### Database Roles and Connections
| Connection Variable | Target Database | Responsibility |
|:---|:---|:---|
| `DATABASE_URL` | Barprep Exam DB | Exam content, practice sessions, drafts, submissions, and "shadow" users. |
| `AUTH_DATABASE_URL` | SHEP Core DB | Centralized user identity, Auth.js PostgresAdapter targets, and cross-product registration. |
Sources: [CLAUDE.md:67-72](), [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql:20-22]()
### System Data Flow
The following diagram illustrates how the application directs traffic between the two databases based on the operation type.
```mermaid
flowchart TD
subgraph SvelteKit_App [Barprep Application]
AuthJS[Auth.js Adapter]
AppLogic[Application Logic]
RegEnd[Registration Endpoint]
end
subgraph SHEP_Core_DB [SHEP Core Database]
UsersTable[Users Table]
IdentityData[Identity / Auth Data]
end
subgraph Barprep_DB [Barprep Database]
ShadowUsers[Shadow Users Table]
ExamData[Sessions / Drafts / Tasks]
end
AuthJS -- writes to --> UsersTable
RegEnd -- writes to --> UsersTable
AppLogic -- queries --> ExamData
AppLogic -- links to --> ShadowUsers
ShadowUsers -. cross-db FK .-> UsersTable
```
A visual representation of the separation between identity management in the SHEP Core DB and exam logic in the Barprep DB.
Sources: [CLAUDE.md:65-80](), [HANDOFF.md:95-100]()
## The Shadow User Pattern
To maintain data integrity without enforcing strict cross-database foreign keys, the architecture employs a "Shadow User" pattern.
1. **Identity Creation:** When a user authenticates via OAuth or registers, Auth.js writes the primary identity record to the SHEP core database.
2. **Shadow Record:** Barprep maintains a local `users` table in the Barprep database. This table contains a `auth_user_id` which acts as a link to the SHEP core user.
3. **FK Enforcement:** Within the Barprep database, all tables (sessions, drafts, submissions) use a standard Foreign Key to the local "shadow" `users.id`. The link from `users.auth_user_id` to the SHEP core is unenforced but guaranteed by the application's authentication flow.
Sources: [CLAUDE.md:77-81](), [apps/web-svelte/database/migrations/001_initial_schema.sql:43-57]()
### Shadow User Schema Definition
```sql
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-- Links to SHEP core's users table (cross-database, unenforced FK).
auth_user_id UUID NOT NULL UNIQUE,
email TEXT NOT NULL,
name TEXT,
exam_type exam_type,
subscription_status subscription_status NOT NULL DEFAULT 'TRIAL',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:43-54]()
## Data Isolation and Polymorphism
The Barprep database is designed for multi-exam support, utilizing polymorphic relationships to handle both NextGen and MEE exam types within shared session and draft tables.
### Shared and Isolated Tables
The architecture distinguishes between shared tables that facilitate common features and domain-specific tables.
* **Shared Tables:** `users`, `practice_sessions`, `drafts`, and `payments`.
* **NextGen Isolated:** `nextgen_tasks`, `nextgen_legal_materials`, `nextgen_submissions`.
* **MEE Isolated:** `mee_essays`, `mee_model_answers`, `mee_submissions`.
Sources: [PRODUCT_SPEC.md:94-105](), [apps/web-svelte/database/migrations/001_initial_schema.sql:60-120]()
### Relationship Diagram (ERD)
The following diagram shows how the polymorphic `practice_sessions` table links to disparate exam content while remaining tied to the shadow user.
```mermaid
erDiagram
SHEP_CORE_USERS ||--o| BARPREP_USERS : "auth_user_id (unenforced)"
BARPREP_USERS ||--o{ PRACTICE_SESSIONS : "user_id"
BARPREP_USERS ||--o{ DRAFTS : "user_id"
PRACTICE_SESSIONS ||--o| NEXTGEN_TASKS : "nextgen_task_id"
PRACTICE_SESSIONS ||--o| MEE_ESSAYS : "mee_essay_id"
NEXTGEN_TASKS ||--o{ NEXTGEN_TASK_MATERIALS : "task_id"
MEE_ESSAYS ||--o| MEE_MODEL_ANSWERS : "essay_id"
```
The ER diagram illustrating the polymorphic links between sessions and specific exam types (NextGen vs MEE).
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:60-150](), [apps/web-svelte/database/schema.sql:400-500]()
## Implementation Details
### Database Connection and Access
The application defines two distinct server-side database utility files:
* `$lib/server/db.ts`: Standard connection for the Barprep Exam database.
* `$lib/server/auth-db.ts`: Specialized connection for the SHEP Core database, primarily used by Auth.js and registration logic.
Sources: [CLAUDE.md:68-72](), [HANDOFF.md:120-125]()
### Constraints and Validation
To prevent data corruption in the polymorphic tables, the schema implements `CHECK` constraints to ensure a session or draft only references one exam type at a time.
```sql
CONSTRAINT valid_task_reference CHECK (
(exam_type = 'NEXTGEN' AND nextgen_task_id IS NOT NULL AND mee_essay_id IS NULL) OR
(exam_type = 'MEE' AND mee_essay_id IS NOT NULL AND nextgen_task_id IS NULL)
)
```
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:75-78]()
## Conclusion
The Dual-Database Architecture provides a robust foundation for Barprep by SHEP, enabling high-performance, isolated exam logic while maintaining a unified user experience with the parent product. This separation of concerns allows the development of specialized grading engines (like the V3.2 engine) and complex polymorphic data structures without impacting the stability of the core SHEP identity system.
Sources: [CLAUDE.md:65-81](), [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql:10-25]()
### SvelteKit State Management Patterns
Relevant source files
The following files were used as context for generating this wiki page:
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
- [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/+page.svelte)
- [apps/web-svelte/src/routes/mee/practice/\[essayId\]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/%5BessayId%5D/+page.svelte)
- [apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte)
- [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/+page.server.ts)
# SvelteKit State Management Patterns
This page outlines the state management architecture for the Bar Prep by SHEP project. The application utilizes Svelte 5 features, specifically "runes" like `$state`, `$derived`, and `$effect`, to manage complex UI states across two primary exam domains: NextGen UBE and MEE.
The project follows a tiered state strategy where global, non-user-specific states (like themes) live in module-level stores, while ephemeral, instance-specific states (like practice sessions) are managed via factory functions to ensure isolation.
Sources: [CLAUDE.md:1-60](), [HANDOFF.md:100-110]()
## Core State Patterns
The application categorizes state based on its scope and persistence requirements. This categorization determines whether state is managed via Svelte 5 runes, SvelteKit's standard data loading, or external persistence layers like local storage and PostgreSQL.
### State Scope Table
| Scope | Implementation Pattern | Usage Example |
| :--- | :--- | :--- |
| **Global/Module** | Module-level `$state` in `.svelte.ts` files | Theme (light/dark), Device/Breakpoint detection |
| **Instance/Context** | Factory functions (`createPracticeState`) | Practice editor split-pane width, focus mode |
| **Server/Auth** | SvelteKit `locals` and `PageData` | User identity, active session IDs, loaded task content |
| **Persistent** | `localStorage` + `$effect` | Theme persistence, draft auto-saving |
Sources: [CLAUDE.md:40-55](), [HANDOFF.md:105-115]()
## Practice Session Management
Practice sessions for NextGen and MEE tasks utilize a factory function pattern called `createPracticeState`. This ensures that each task instance has a fresh, isolated state object, preventing state leakage between different practice attempts.
### Data Flow and Persistence
The application uses Svelte `$effect` blocks to synchronize in-memory state with the backend. For example, during a practice session, a timer triggers a background `fetch` to a save API every 30 seconds to persist user drafts.
```mermaid
flowchart TD
A[Page Load] --> B[Load Data from +page.server.ts]
B --> C[Initialize createPracticeState]
C --> D{User Action}
D -->|Typing| E[Update $state content]
D -->|Toggle| F[Update UI Mode Focus/Mobile]
E --> G[$effect Auto-save]
G -->|Every 30s| H[PUT /api/sessions/id/save]
H --> I[PostgreSQL Database]
```
The state management logic differentiates between "raw" content (the student's writing) and UI configuration (panel widths and modes).
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:60-100](), [apps/web-svelte/src/routes/mee/practice/[essayId]/+page.svelte:40-70]()
## Svelte 5 Rune Utilization
The project strictly adheres to Svelte 5 conventions to handle reactivity and component communication.
* **`$state`**: Used for mutable values such as the current content of an essay, the visibility of the timer, or whether a user is in "Focus Mode".
* **`$derived`**: Used for computed values, such as calculating the `totalSeconds` from minutes or determining if a task is a "demo" task.
* **`$effect`**: Employed for side effects, primarily for synchronization between Svelte state and external APIs or the DOM (e.g., applying CSS classes to `document.documentElement`).
* **`$props()`**: Replaces the old `export let` syntax for passing data into components with TypeScript interfaces.
Sources: [CLAUDE.md:75-85](), [apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte:25-45]()
## Theme and Device Stores
Global UI state that is not specific to a single user's exam data is stored in specialized stores.
### Theme Store
Manages light/dark/night modes and "paper" variants. It utilizes an inline script in `app.html` to prevent Flash of Unstyled Content (FOUC) by reading from `localStorage` before the first paint.
### Device Store
Provides reactive access to viewport information, such as `device.isDesktop`. This is used to conditionally render different layouts, such as the split-pane view on desktop versus a tabbed view on mobile.
```mermaid
sequenceDiagram
participant LS as LocalStorage
participant APP as app.html Script
participant TS as theme.svelte.ts
participant DOM as Document Element
LS->>APP: Get barprep_theme
APP->>DOM: Add .dark class
APP->>DOM: Set data-theme variant
Note over TS: Svelte App Hydrates
TS->>LS: Sync $state with storage
```
Sources: [apps/web-svelte/src/app.html:20-45](), [CLAUDE.md:42-45](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:180-220]()
## Server-Side State Integration
SvelteKit's `+page.server.ts` acts as the bridge between the persistent database and the reactive frontend state.
1. **Load Function**: Retrieves the `activeSession`, `draft`, and `task` data. This is passed to the page as `data`.
2. **Actions**: Handles transitions like `start` (creating a new session) and `submit` (marking a session as completed).
3. **Authentication**: Flowing through `event.locals.auth()`, user state is used to scope all session and draft lookups to the specific `barprepUserId`.
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts:15-50](), [CLAUDE.md:46-50]()
## Summary
The state management in Bar Prep by SHEP leverages Svelte 5's fine-grained reactivity to handle the complex requirements of a bar exam simulator. By separating non-user global state (Theme/Device) from instance-specific practice state (Factory Functions) and utilizing SvelteKit's server-side loading for persistent data, the architecture ensures a performant and reliable user experience across different devices and exam types.
## Core Features
### User Authentication & Onboarding
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/auth.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/auth.ts)
- [apps/web-svelte/src/hooks.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/hooks.server.ts)
- [apps/web-svelte/src/routes/login/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/login/%2Bpage.svelte)
- [apps/web-svelte/PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
# User Authentication & Onboarding
The User Authentication and Onboarding system in Bar Prep by SHEP is designed as a standalone extension of the SHEP core product. It implements a dual-database architecture where user identity is managed globally via the SHEP core database, while application-specific progress, exam types, and subscription states are maintained in a local "shadow" database. This ensures that Bar Prep users are integrated into the broader SHEP ecosystem for unified management while keeping exam-specific data isolated.
Sources: [CLAUDE.md](), [HANDOFF.md:14-18](), [PRODUCT_SPEC.md:15-18]()
## Authentication Architecture
The system utilizes **Auth.js** (`@auth/sveltekit`) to manage authentication across multiple providers, including Google OAuth and traditional Email/Password credentials. The architecture is characterized by its "Dual-Database" strategy:
* **SHEP Core Database (`AUTH_DATABASE_URL`):** Acts as the primary identity provider. The `PostgresAdapter` writes to this database so that users appear in the core SHEP user count.
* **Barprep Local Database (`DATABASE_URL`):** Stores "shadow" user records that link back to the core identity via an `auth_user_id`. This local table contains bar-prep specific metadata such as `exam_type` and `subscription_status`.
Sources: [CLAUDE.md](), [apps/web-svelte/src/lib/server/auth.ts:1-20]()
### Database Linkage
The relationship between the primary identity and the local shadow record is managed during the JWT callback. If a local record does not exist when a user signs in, one is created (upserted) to ensure synchronization.
```mermaid
erDiagram
CORE_USERS ||--o| BARPREP_USERS : "auth_user_id (unenforced)"
BARPREP_USERS ||--o{ PRACTICE_SESSIONS : "user_id"
BARPREP_USERS ||--o{ DRAFTS : "user_id"
BARPREP_USERS ||--o{ SUBMISSIONS : "user_id"
BARPREP_USERS {
uuid id PK
uuid auth_user_id UK
string email
exam_type exam_type
sub_status subscription_status
}
```
*Note: The `auth_user_id` serves as a cross-database foreign key that is unenforced at the database level but guaranteed by the Auth.js flow.*
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:47-60](), [CLAUDE.md]()
## Login and Registration Flow
Users interact with a unified login interface located at `/login`. This page supports toggling between sign-in and registration modes.
### Authentication Providers
| Provider | Implementation Details |
| :--- | :--- |
| **Google OAuth** | Configured via `AUTH_GOOGLE_ID` and `AUTH_GOOGLE_SECRET`. Triggered by `signIn('google')`. |
| **Credentials** | Uses `bcryptjs` for password hashing and comparison. Validates against the `users` table in the SHEP core database. |
Sources: [apps/web-svelte/src/lib/server/auth.ts:39-80](), [apps/web-svelte/src/routes/login/+page.svelte:1-35]()
### Security Measures
To prevent user enumeration, the `authorize` callback for credentials implements a timing-safe dummy hash comparison. If a user is not found, `bcrypt.compare` is still executed against a fixed dummy hash to ensure consistent response times.
Sources: [apps/web-svelte/src/lib/server/auth.ts:21-23](), [apps/web-svelte/src/lib/server/auth.ts:68-75]()
## Server Hooks and Protection
Authentication and onboarding state are enforced via SvelteKit server hooks in `hooks.server.ts`. The process follows a sequence of states to determine access.
### Route Protection Logic
1. **Public Path Check:** Routes like `/`, `/login`, and `/api/auth` are bypassed.
2. **Authentication Check:** If no session is found, users are redirected to `/login`.
3. **Shadow Sync Check:** Verifies that a `barprepUserId` exists in the local database. If missing, it indicates a service failure or sync issue.
4. **Onboarding Gate:** If `exam_type` is null, the user is redirected to `/onboarding`.
5. **Cross-Exam Blocking:** Users selected for 'NEXTGEN' cannot access '/mee' routes and vice-versa.
```mermaid
flowchart TD
Req[Incoming Request] --> Public{Is Public Path?}
Public -- Yes --> Resolve[Resolve Request]
Public -- No --> Auth{Authenticated?}
Auth -- No --> Login[Redirect to /login]
Auth -- Yes --> Sync{Shadow User Exists?}
Sync -- No --> Error[Redirect to /login?error=ServiceUnavailable]
Sync -- Yes --> Onboarded{Exam Type Selected?}
Onboarded -- No --> Onboarding[Redirect to /onboarding]
Onboarded -- Yes --> ExamCheck{Correct Exam Path?}
ExamCheck -- No --> RedirectExam[Redirect to valid exam root]
ExamCheck -- Yes --> Final[Resolve Request]
```
Sources: [apps/web-svelte/src/hooks.server.ts:18-85]()
## User Onboarding & Subscription
Upon first successful login, users are moved into the onboarding state. This is the only phase where the `exam_type` field in the `users` table is `NULL`.
### Exam Type Selection
Users must choose between:
* **NEXTGEN:** The skills-based bar exam (SPT, LRPT, IQS).
* **MEE:** The traditional Multistate Essay Examination.
### Subscription Status
New users are assigned a `TRIAL` status by default. The system is designed to support a subscription model at $299 per bar prep cycle, although gating is currently disabled for demo navigation in some contexts.
| Field | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `exam_type` | ENUM | NULL | 'NEXTGEN' or 'MEE' |
| `subscription_status` | ENUM | 'TRIAL' | 'ACTIVE', 'EXPIRED', or 'TRIAL' |
| `subscription_expires_at` | TIMESTAMPTZ | NULL | Expiration date of access |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:19-22](), [apps/web-svelte/database/migrations/001_initial_schema.sql:47-60](), [PRODUCT_SPEC.md:7-13]()
## Summary
The User Authentication & Onboarding system provides a bridge between the SHEP core identity and Bar Prep specific functionality. By leveraging SvelteKit hooks and a shadow database, it maintains strict separation of concerns while ensuring a seamless user experience that guides candidates from initial sign-up to their specific exam dashboard.
Sources: [HANDOFF.md:104-106](), [CLAUDE.md]()
### Subscription & Stripe Payments
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/routes/api/stripe/webhook/+server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/api/stripe/webhook/%2Bserver.ts)
- [apps/web-svelte/src/routes/checkout/success/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/checkout/success/%2Bpage.server.ts)
- [apps/web-svelte/database/migrations/005_stripe_fields.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/005_stripe_fields.sql)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
# Subscription & Stripe Payments
## Introduction
The Subscription & Stripe Payments system handles the monetization and access control for the Bar Prep by SHEP platform. It implements a fixed-price model where users pay $299 per bar prep cycle (approximately 3–4 months or up to 6 months of access) to unlock practice features for either the NextGen UBE or MEE exam tracks. The system is designed to integrate with Stripe for transaction processing and uses a "shadow user" model to link payments to the core SHEP identity system.
Sources: [PRODUCT_SPEC.md:12-16](), [HANDOFF.md:13-16](), [apps/web-svelte/src/routes/api/stripe/webhook/+server.ts:46-47]()
## Data Model
The payment architecture relies on a specialized `payments` ledger and extensions to the `users` table. The schema ensures idempotency by tracking Stripe Session IDs to prevent duplicate processing of the same transaction.
### Database Schema
| Table | Field | Type | Description |
| :--- | :--- | :--- | :--- |
| `users` | `stripe_customer_id` | TEXT | Unique identifier for the customer in Stripe. |
| `users` | `subscription_status` | ENUM | Status of access (ACTIVE, EXPIRED, TRIAL). |
| `users` | `subscription_expires_at` | TIMESTAMPTZ | Expiration date of the current cycle. |
| `payments` | `stripe_session_id` | TEXT | Unique Stripe Checkout Session ID (indexed). |
| `payments` | `amount_cents` | INTEGER | Transaction amount in the smallest currency unit. |
| `payments` | `exam_type` | ENUM | The exam track purchased (NEXTGEN or MEE). |
Sources: [apps/web-svelte/database/migrations/005_stripe_fields.sql:17-31](), [apps/web-svelte/database/schema.sql:351-360](), [apps/web-svelte/database/schema.sql:684-694]()
### Entity Relationships
The following diagram illustrates how payment records link users to specific transactions and exam types.
```mermaid
erDiagram
USERS ||--o{ PAYMENTS : owns
USERS {
uuid id PK
text stripe_customer_id UK
subscription_status status
timestamptz expires_at
}
PAYMENTS {
uuid id PK
text stripe_session_id UK
text stripe_payment_intent_id
integer amount_cents
exam_type exam_type
text status
}
```
Diagram shows the relationship between User records and the Payment ledger.
Sources: [apps/web-svelte/database/schema.sql:351-360](), [apps/web-svelte/database/schema.sql:684-694](), [apps/web-svelte/database/migrations/005_stripe_fields.sql:17-31]()
## Payment Processing Flow
The system utilizes Stripe Checkout for handling transactions. Post-payment, the application confirms the state through two primary channels: a client-side success redirect and a server-to-server webhook.
### Checkout Success Handling
Upon successful completion of a Stripe Checkout session, the user is redirected to `/checkout/success`. The server-side load function verifies the session status directly with Stripe before updating the local database.
```mermaid
sequenceDiagram
participant U as User Browser
participant S as Stripe API
participant APP as App Server (+page.server.ts)
participant DB as Neon Database
U->>APP: GET /checkout/success?session_id=...
APP->>S: sessions.retrieve(sessionId)
S-->>APP: session (payment_status: 'paid')
APP->>DB: SELECT subscription_status FROM users
alt Status not ACTIVE
APP->>DB: UPDATE users SET status='ACTIVE', expires_at=...
APP->>DB: INSERT INTO payments (...)
end
APP-->>U: Return success (trigger JWT refresh)
```
Sequence of events during the post-payment redirect flow.
Sources: [apps/web-svelte/src/routes/checkout/success/+page.server.ts:16-58]()
### Webhook Integration
The webhook endpoint at `/api/stripe/webhook` provides an asynchronous, reliable method for updating subscription statuses. It handles the `checkout.session.completed` event and uses metadata passed during session creation (specifically `barprep_user_id` and `exam_type`) to locate the correct user record.
Key logic includes:
1. **Signature Verification**: Validates requests using the `STRIPE_WEBHOOK_SECRET`.
2. **Idempotency**: Checks if the user's status is already `ACTIVE` before performing updates.
3. **Ledger Entry**: Records the `amount_total` and `payment_intent` into the `payments` table.
Sources: [apps/web-svelte/src/routes/api/stripe/webhook/+server.ts:11-37](), [apps/web-svelte/src/routes/api/stripe/webhook/+server.ts:46-75]()
## Subscription Lifecycle Management
The system enforces access control via the `subscription_status` enum. Users typically start in a `TRIAL` state and transition to `ACTIVE` upon payment.
### Lifecycle Logic
* **Activation**: When a payment is confirmed, the user's status is set to `ACTIVE` and an expiration date is calculated (typically 6 months from the current date).
* **Expiration**: The `subscription_expires_at` field is used to determine when access to practice tasks should be revoked.
* **Cycle-Based Pricing**: Access is restricted by "cycle," meaning users select their exam type (NextGen or MEE) at signup or during the purchase flow.
Sources: [apps/web-svelte/src/routes/api/stripe/webhook/+server.ts:48-55](), [PRODUCT_SPEC.md:12-16](), [apps/web-svelte/database/schema.sql:351-360]()
## Configuration
Environmental variables required for the Stripe integration:
| Variable | Source | Description |
| :--- | :--- | :--- |
| `STRIPE_WEBHOOK_SECRET` | Private Env | Used to verify the authenticity of Stripe webhook events. |
| `DATABASE_URL` | Private Env | Primary connection for updating user and payment records. |
Sources: [apps/web-svelte/src/routes/api/stripe/webhook/+server.ts:6-16](), [HANDOFF.md:61-65]()
## Summary
The Subscription & Stripe Payments system provides a robust framework for managing bar prep access. By combining real-time Stripe session verification with an idempotent webhook handler, the system ensures that user access is reliably granted upon payment of the $299 fee. The architectural split between the `users` metadata and the `payments` ledger allows for detailed financial tracking while maintaining a clean user state for the SvelteKit frontend.
### The Chambers (Practice Editor)
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/+page.svelte)
- [apps/web-svelte/src/lib/nextgen/components/MaterialsPanel.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/MaterialsPanel.svelte)
- [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/+page.server.ts)
- [apps/web-svelte/src/lib/nextgen/components/AssignmentMemo.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/+page.svelte) (referenced via import)
- [apps/web-svelte/src/lib/components/exam/RichTextEditor.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/components/exam/RichTextEditor.svelte)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
# The Chambers (Practice Editor)
The Chambers is the core practice environment within the Bar Prep by SHEP application, specifically designed for the **NextGen UBE** and **MEE** examination tracks. It provides a high-fidelity, split-pane writing interface that emulates the official NCBE examination experience, allowing users to analyze legal materials and draft memoranda or essays under simulated exam conditions.
The system integrates several specialized components including a responsive split-pane layout, a legal materials library, a synchronized practice timer, and a rich text editor with auto-save capabilities. Its primary goal is to facilitate structured legal reasoning practice—specifically addressing "The Chambers" for NextGen Performance Tasks (SPT/LRPT) and standard MEE essay sessions.
Sources: [HANDOFF.md:38-46](), [PRODUCT_SPEC.md:58-62](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:150-160]()
## Architecture and Core Components
The Chambers is implemented as a Svelte 5 application using runes for state management. The architecture centers around a "Practice State" factory that manages the lifecycle of a practice session, including layout dimensions, focus modes, and content persistence.
### Primary UI Components
| Component | Responsibility |
| :--- | :--- |
| `AssignmentMemo` | Displays the initial law firm supervisor memo and role prompt before the session begins. |
| `MaterialsPanel` | A tabbed interface providing access to the "File" (case facts) and "Library" (legal authorities). |
| `RichTextEditor` | The primary input area for drafting legal analysis, supporting markdown-style formatting. |
| `PracticeTimer` | A visual progress bar that tracks elapsed time, provides threshold warnings, and handles time-up events. |
| `SplitPane` | Manages the draggable divider between the materials and the writing surface. |
Sources: [HANDOFF.md:41-46](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:11-20](), [apps/web-svelte/src/lib/nextgen/components/MaterialsPanel.svelte:1-55]()
### Layout and State Flow
The editor supports a "Focus Mode" which maximizes the writing surface by hiding the materials panel, and a responsive mobile layout that switches from a split-pane view to a tabbed navigation (Materials vs. Write).
```mermaid
graph TD
User[User] -->|Starts Session| Memo[Assignment Memo]
Memo -->|On Start| Editor[The Chambers Editor]
subgraph Editor_Layout [Editor Interface]
Timer[Practice Timer]
Split[Split Pane Controller]
Materials[Materials Panel]
Write[Rich Text Editor]
end
Editor --> Timer
Editor --> Split
Split --> Materials
Split --> Write
Write -->|30s Interval| AutoSave[Auto-save API]
Write -->|Submit| API[Submission Endpoint]
```
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:66-88](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:176-190]()
## Legal Materials Management
The `MaterialsPanel` serves as the research hub for the user. It is divided into two primary contexts via a tabbed interface.
### Materials Contexts
1. **The File:** Contains scenario-specific documents such as the `Statement of Facts`, `Background` information, and `Client Documents` (correspondence, exhibits, etc.).
2. **The Library:** Houses legal authorities provided for the task, including `Statutes`, `Case Excerpts`, `Regulations`, and `Restatements`.
### Data Structure for Materials
Materials are categorized by `MaterialType` and sorted based on a `sort_order` attribute. Content is sanitized using `isomorphic-dompurify` before being rendered as HTML.
```typescript
type MaterialType = 'statute' | 'case' | 'case_excerpt' | 'regulation' | 'client_document' | 'document';
interface Material {
id: string;
title: string;
content: string;
citation?: string;
type?: MaterialType;
sort_order?: number;
}
```
Sources: [apps/web-svelte/src/lib/nextgen/components/MaterialsPanel.svelte:5-30](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:25-32]()
## Data Persistence and Session Lifecycle
The editor maintains session integrity through a combination of server-side loading and client-side auto-saving.
### Session Transitions
The session lifecycle is managed through SvelteKit form actions and API endpoints:
- **Start:** Initiates a `practice_session` in the database with a status of `IN_PROGRESS`.
- **Save:** A client-side `$effect` triggers a `PUT` request to `/api/sessions/[sessionId]/save` every 30 seconds if the session is active.
- **Submit:** Sanitizes the HTML content using an allow-list and creates a `nextgen_submission` (or `mee_submission`), updating the session status to `COMPLETED`.
- **Exit:** Provides options to "Save and Exit" (preserving the draft) or "Discard and Exit" (abandoning the session).
```mermaid
sequenceDiagram
participant U as User
participant C as Client (The Chambers)
participant S as SvelteKit Server
participant DB as Neon Database
U->>C: Click "Begin Task"
C->>S: POST ?/start
S->>DB: Create Session (IN_PROGRESS)
DB-->>S: Session ID
S-->>C: Load Editor State
loop Every 30 Seconds
C->>S: PUT /api/sessions/:id/save (Content + Elapsed Time)
S->>DB: Update Drafts Table
end
U->>C: Click "Submit Analysis"
C->>S: POST ?/submit (Sanitized Content)
S->>DB: Create Submission & Set Session COMPLETED
S-->>C: Show Results/Success
```
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:66-105](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts:41-91]()
## User Interface Specifications
The UI utilizes **Tailwind CSS 4** and the **SHEP Design System** tokens to ensure a professional, scholarly environment.
| Feature | Implementation Detail | Source |
| :--- | :--- | :--- |
| **Theme Support** | Light, Dark, Paper, and Night variants. | `HANDOFF.md:20` |
| **Typography** | Source Serif 4 (Body), Cormorant Garamond (Display), Geist (UI). | `HANDOFF.md:104` |
| **Draggable Divider** | A `divider-gap` with `cursor-col-resize` and keyboard nudge support (Arrow keys). | `[taskId]/+page.svelte:350-370` |
| **Focus Mode** | Hides the left materials panel and sets the writing surface width to 100%. | `[taskId]/+page.svelte:200-220` |
| **Responsive Design** | Mobile view uses a tabbed layout (`mobileTab` state) instead of split-pane. | `[taskId]/+page.svelte:235-265` |
### Code Example: Keyboard Navigation for Resize
```javascript
function handleDividerKeydown(e: KeyboardEvent) {
if (e.key === 'ArrowLeft') {
e.preventDefault();
practiceState.nudgePanelWidth(-practiceState.KEYBOARD_NUDGE_STEP);
} else if (e.key === 'ArrowRight') {
e.preventDefault();
practiceState.nudgePanelWidth(practiceState.KEYBOARD_NUDGE_STEP);
}
}
```
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:137-145]()
## Conclusion
The Chambers (Practice Editor) represents a sophisticated integration of state management and UI components designed to mimic high-stakes testing environments. By leveraging a dual-database architecture for authentication and exam data, it provides a seamless, auto-saving experience that ensures user progress is maintained across sessions while providing all necessary legal materials in a structured, accessible layout.
Sources: [CLAUDE.md:38-50](), [HANDOFF.md:40-50]()
### NextGen Practice Flow
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/%2Bpage.svelte)
- [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/%2Bpage.server.ts)
- [apps/web-svelte/src/lib/nextgen/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/types.ts)
- [apps/web-svelte/src/routes/nextgen/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/%2Bpage.svelte)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
# NextGen Practice Flow
The NextGen Practice Flow is a specialized instructional and examination environment designed to simulate the **NextGen UBE** (launching July 2026). It focuses on skills-based legal reasoning through specific task types: Standard Performance Tasks (SPT), Legal Research Performance Tasks (LRPT), and Integrated Question Sets (IQS). Sources: [PRODUCT_SPEC.md:8-12](), [HANDOFF.md:12-14]()
The system utilizes a "Chambers" interface—a split-pane writing environment that integrates legal materials, supervisor memos, and a rich text editor. This flow ensures that candidates practice under realistic exam conditions with features like persistent timers, auto-saving drafts, and official NCBE-style formatting. Sources: [PRODUCT_SPEC.md:46-52](), [HANDOFF.md:44-50]()
## Task Architecture and Types
NextGen practice is structured around three primary task categories, each testing different legal competencies. These tasks are characterized by the presence of a "Case File" or "Library" of legal sources. Sources: [apps/web-svelte/src/lib/nextgen/types.ts:1-12](), [PRODUCT_SPEC.md:20-28]()
| Task Type | Description | Exam Weight |
| :--- | :--- | :--- |
| **SPT** | Standard Performance Task: Read materials, write memo/brief | ~35% (combined with LRPT) |
| **LRPT** | Legal Research Performance Task: Researching and drafting | ~35% (combined with SPT) |
| **IQS** | Integrated Question Set: Scenario-based MCQ and short answers | ~25% |
Sources: [apps/web-svelte/src/routes/nextgen/+page.svelte:85-135](), [apps/web-svelte/src/lib/nextgen/types.ts:1-3]()
## Practice Session Lifecycle
The practice flow follows a strict state progression: Initialization, Active Engagement (The Chambers), and Submission. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:115-180]()
### 1. Initialization and Assignment Memo
Before the timer begins, users are presented with an **Assignment Memo**. This component provides the role prompt, supervisor instructions, and a list of attached documents. The session officially transitions to "Active" only when the user clicks the start action, which triggers a server-side session creation. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:170-182](), [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.server.ts:46-68]()
### 2. The Chambers (Active Practice)
Once started, the interface enters a split-pane mode. The left panel contains the **MaterialsPanel**, displaying statutes, cases, and client documents. The right panel contains the **RichTextEditor**. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:185-230](), [HANDOFF.md:44-48]()
### 3. State Persistence and Auto-save
To prevent data loss, the system implements an auto-save mechanism that triggers every 30 seconds. This updates the `practice_sessions` table with elapsed time and the `drafts` table with current content. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:57-78](), [apps/web-svelte/database/migrations/001_initial_schema.sql:81-100]()
The following diagram illustrates the transition between these states:
```mermaid
flowchart TD
Start[User selects Task] --> Memo[Assignment Memo View]
Memo -->|POST ?/start| Active[Active Practice: The Chambers]
Active -->|30s Interval| AutoSave[Background Auto-save]
AutoSave --> Active
Active -->|User clicks Exit| Confirm{Exit Dialog}
Confirm -->|Save & Exit| Dashboard[NextGen Dashboard]
Confirm -->|Discard| Dashboard
Active -->|POST ?/submit| Submit[Submission View]
```
The flow ensures that sessions can be resumed if a user exits prematurely without discarding. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:84-110](), [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.server.ts:25-44]()
## Data Models and Schema
The NextGen Practice Flow relies on a polymorphic database structure to handle different exam types while maintaining shared session tracking. Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:53-79]()
### Practice Sessions Table
The `practice_sessions` table tracks the progress of a user's attempt.
| Field | Type | Description |
| :--- | :--- | :--- |
| `id` | UUID | Primary key for the session |
| `user_id` | UUID | Reference to local shadow user |
| `exam_type` | ENUM | Must be 'NEXTGEN' for this flow |
| `nextgen_task_id` | UUID | Foreign key to `nextgen_tasks` |
| `status` | ENUM | IN_PROGRESS, COMPLETED, ABANDONED, TIMED_OUT |
| `elapsed_seconds` | INT | Accumulated time spent |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:53-79](), [apps/web-svelte/src/lib/nextgen/types.ts:47-58]()
### Legal Materials
Materials are linked to tasks via a junction table that supports **Progressive Disclosure** (used primarily in IQS flows) via the `disclosure_step` field. Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:133-143]()
```mermaid
erDiagram
NEXTGEN_TASKS ||--o{ NEXTGEN_TASK_MATERIALS : contains
NEXTGEN_LEGAL_MATERIALS ||--o{ NEXTGEN_TASK_MATERIALS : used_in
PRACTICE_SESSIONS ||--o| NEXTGEN_TASKS : references
PRACTICE_SESSIONS ||--o{ NEXTGEN_SUBMISSIONS : creates
USERS ||--o{ PRACTICE_SESSIONS : owns
NEXTGEN_TASKS {
uuid id
text title
nextgen_task_type task_type
jsonb supervisor_memo
}
NEXTGEN_LEGAL_MATERIALS {
uuid id
text title
text content
nextgen_material_type material_type
}
```
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:106-157]()
## Implementation Details
### Session Resumption Logic
The `load` function in the SvelteKit server route checks for an existing active session and draft. If found, it populates the client-side `practiceState` with the previous work. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.server.ts:21-44]()
```typescript
// From apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts
const [activeSession, draft] = await Promise.all([
getActiveSession(barprepUserId, { nextgenTaskId: taskId }),
getDraft(barprepUserId, { nextgenTaskId: taskId })
]);
```
### UI Components
* **PracticeTimer**: Manages countdown/count-up logic, thresholds warnings, and visibility toggles. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:189-196](), [HANDOFF.md:32]()
* **MaterialsPanel**: Implements tabbed navigation (File/Library) for sorting through legal authorities. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:225-230](), [HANDOFF.md:34]()
* **RichTextEditor**: A custom toolbar-equipped editor that supports legal markdown and IRAC scaffolding. Sources: [apps/web-svelte/src/routes/nextgen/practice/\[taskId\]/+page.svelte:264-270](), [HANDOFF.md:33]()
## Conclusion
The NextGen Practice Flow provides a robust, state-managed environment for bar preparation. By integrating a detailed schema for legal materials with a high-fidelity interface ("The Chambers"), the system allows for accurate simulation of new skills-based bar examination formats while ensuring data persistence through background auto-saving and polymorphic session tracking. Sources: [HANDOFF.md:12-20](), [PRODUCT_SPEC.md:107-115]()
### MEE Practice Flow
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/routes/mee/practice/\[essayId\]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/%5BessayId%5D/%2Bpage.svelte)
- [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/packet/%5BpacketId%5D/%2Bpage.svelte)
- [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/packet/%5BpacketId%5D/%2Bpage.server.ts)
- [apps/web-svelte/src/lib/exam-types/mee/constants.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/mee/constants.ts)
- [apps/web-svelte/src/lib/exam-types/mee/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/mee/types.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
# MEE Practice Flow
The Multistate Essay Examination (MEE) Practice Flow is a core module within Bar Prep by SHEP designed to simulate the traditional essay-based bar exam format. It provides law students with a structured environment to practice issue spotting and IRAC-based legal writing across 13 subject areas.
The flow transitions from a pre-session "Reading" state where the fact pattern is presented, through an active "In-Session" desk environment with timed or untimed writing, to a final submission and automated grading phase. The system emphasizes a "physical workspace" aesthetic, separating source materials from the student's legal pad.
Sources: [PRODUCT_SPEC.md:14-22](), [apps/web-svelte/src/routes/mee/practice/\[essayId\]/+page.svelte:353-358](), [apps/web-svelte/src/lib/exam-types/mee/types.ts:6-21]()
## User Workflow
The practice experience is divided into three distinct lifecycle stages: Pre-Session, Active Session, and Submission/Grading.
### Pre-Session and Initialization
Before starting, users review the essay metadata (subject, difficulty, and time limit) and the fact pattern. They must select a timer mode before beginning the attempt.
```mermaid
flowchart TD
Start([User Selects Essay]) --> Load[Load Essay Data]
Load --> PreSession[Display Fact Pattern & Metadata]
PreSession --> ModeSelect{Select Timer Mode}
ModeSelect --> Timed[30 Min Timed]
ModeSelect --> Untimed[Untimed]
Timed --> Begin[Click 'Begin Essay']
Untimed --> Begin
Begin --> Active[Start Practice Session]
```
The application defaults to a 30-minute time limit for timed sessions. Sources: [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.svelte:121-164](), [apps/web-svelte/src/lib/exam-types/mee/constants.ts:6]()
### Active Session "The Desk"
The active session utilizes a "Desk" interface. On desktop, this is a side-by-side view where the left column serves as the "case file" (fact pattern and call of the question) and the right column serves as the "legal pad" (Rich Text Editor). On mobile, these panels are stacked with a collapsible prompt.
Key features during the active session include:
* **Auto-save:** The system performs a "best-effort" auto-save of the draft content every 30 seconds.
* **Word Count:** A real-time count of words written is displayed in the header.
* **Hysteresis Timer:** A horizon timer that tracks remaining or elapsed time depending on the mode.
Sources: [apps/web-svelte/src/routes/mee/practice/\[essayId\]/+page.svelte:204-219](), [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.svelte:170-205](), [apps/web-svelte/src/routes/mee/practice/\[essayId\]/+page.svelte:44-63]()
### Submission and Grading Pipeline
Upon clicking submit, the UI enters a "Grading Overlay" state while the server executes a multi-stage grading pipeline (v3.2).
```mermaid
sequenceDiagram
participant UI as Browser (Svelte)
participant Srv as SvelteKit Server
participant Runtime as Grading Runtime (v3.2)
participant AI as AI Gateway (LLM)
UI->>Srv: POST /submit (form data)
Srv->>Srv: Normalize text & create submission record
Srv->>Runtime: gradeSubmission()
activate Runtime
Runtime->>AI: Quality Channel (Whole essay analysis)
Runtime->>AI: Embedding (Vectorize sentences)
Runtime->>AI: Packet Adherence (Issue-specific scoring)
Runtime->>Runtime: Deterministic Aggregation & Banding
Runtime->>AI: Feedback Renderer (Generate narrative)
deactivate Runtime
Runtime-->>Srv: GradingRun result
Srv-->>UI: Redirect to Results page
```
Sources: [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.svelte:82-108](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:51-180](), [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.server.ts:133-188]()
## Data Models and Types
### Core Entities
The system uses polymorphic session and draft tables to handle both NextGen and MEE content types.
| Table/Type | Key Fields | Description |
| :--- | :--- | :--- |
| `mee_essays` | `id`, `subject`, `difficulty`, `fact_pattern`, `call_of_the_question` | Metadata and content for an essay prompt. |
| `practice_sessions` | `id`, `user_id`, `status`, `elapsed_seconds`, `mee_essay_id` | Tracks the lifecycle of a user's attempt. |
| `mee_submissions` | `id`, `content`, `word_count`, `rubric_scores`, `total_score` | Final record of a completed essay attempt. |
| `MEEIssueScore` | `issueKey`, `identified`, `ruleStated`, `pointsAwarded` | Component scores for specific legal issues within an essay. |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:71-125](), [apps/web-svelte/src/lib/exam-types/mee/types.ts:46-56]()
### Grading Run Configuration
The v3.2 runtime utilizes an `ExecutableScoringBundle` and a `ModelManifest` to define how an essay is analyzed.
* **Quality Channel:** Analyzes the whole essay for general writing quality.
* **Packet Adherence:** Evaluates the student's performance against specific checklist items for each identified legal issue.
* **Banding:** Calibrates raw scores into a SHEP Practice Band (modeled on MEE 1–6).
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:182-243](), [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.server.ts:153-188]()
## Implementation Details
### State Management
Active practice state is managed via a factory function `createMeePracticeState` (or `createPracticeState` in older iterations), which utilizes Svelte 5 runes for reactivity.
```typescript
// Sample state initialization
const ps = createMeePracticeState({ packetId: data.packet.packet_id });
// Reactivity usage
const timeDisplay = $derived(
ps.timeRemainingSeconds !== null
? formatTime(ps.timeRemainingSeconds)
: formatTime(ps.elapsedSeconds)
);
```
Sources: [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.svelte:18-62](), [apps/web-svelte/src/routes/mee/practice/\[essayId\]/+page.svelte:19-23]()
### Template System
To assist students in structured writing, the flow includes an IRAC (Issue, Rule, Analysis, Conclusion) template system. This can be manually toggled or inserted into the editor.
```markdown
## Issue
[Identify the legal issue(s) presented by the facts]
## Rule
[State the applicable legal rule(s)]
## Analysis
[Apply the rule(s) to the facts]
## Conclusion
[State your conclusion]
```
Sources: [apps/web-svelte/src/lib/exam-types/mee/constants.ts:68-80](), [apps/web-svelte/src/routes/mee/practice/+page.svelte:31-62]()
## Summary
The MEE Practice Flow provides a specialized environment for high-stakes essay training. By utilizing a side-by-side "Desk" architecture and a sophisticated v3.2 AI grading runtime, it enables both realistic exam simulation and immediate, detailed feedback on legal issue spotting and IRAC structure. Sources: [PRODUCT_SPEC.md:46-55](), [apps/web-svelte/src/routes/mee/practice/packet/\[packetId\]/+page.svelte:170-185]()
### MCQ & Question Handling
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/exam-types/nextgen/mcq-types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/nextgen/mcq-types.ts)
- [apps/web-svelte/src/routes/nextgen/mcq/[sessionId]/exam-session-state.svelte.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/mcq/%5BsessionId%5D/exam-session-state.svelte.ts)
- [apps/web-svelte/src/routes/nextgen/mcq/[sessionId]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/mcq/%5BsessionId%5D/%2Bpage.svelte)
- [apps/web-svelte/src/routes/nextgen/mcq/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/mcq/%2Bpage.svelte)
- [apps/web-svelte/src/routes/nextgen/mcq/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/mcq/%2Bpage.server.ts)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
- [apps/web-svelte/src/lib/nextgen/components/exam/McqResults.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/exam/McqResults.svelte)
# MCQ & Question Handling
Multiple-Choice Question (MCQ) handling in Bar Prep by SHEP is a comprehensive system designed to simulate NextGen UBE testing environments. It encompasses question configuration, real-time session state management, persistent response tracking, and detailed performance analytics. The system supports various interaction types, including single-select and multi-select (select-two) questions, as well as features like option strikeouts and "marked for review" flagging.
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/mcq-types.ts](), [apps/web-svelte/src/routes/nextgen/mcq/+page.svelte]()
## Question and Session Data Models
The system relies on a structured data model to distinguish between private administrative data (like correct answers and explanations) and public data served to students during an active session.
### Data Structures
| Type | Description |
|:---|:---|
| `McqQuestionRow` | The full database representation containing the stem, instruction type, difficulty, and options with correct answer flags. |
| `McqQuestionPublic` | A sanitized version of the question safe for client-side transmission, omitting correctness data until the review phase. |
| `McqOption` | Individual answer choice containing the key, text, correctness boolean, and rationales for that choice. |
| `McqSetConfig` | Configuration object for a session, including subjects, question count, difficulty filters, and feedback mode. |
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/mcq-types.ts:31-90]()
### Database Schema Relationships
The following diagram illustrates the relationship between questions, practice sessions, and user responses in the PostgreSQL schema.
```mermaid
erDiagram
mcq_questions ||--o{ mcq_responses : "has many"
practice_sessions ||--o{ mcq_responses : "contains"
users ||--o{ practice_sessions : "starts"
practice_sessions {
uuid id
jsonb mcq_set_config
practice_session_status status
int elapsed_seconds
}
mcq_responses {
uuid id
uuid session_id
text_array selected_options
text_array strikeouts
boolean is_correct
int version
}
mcq_questions {
uuid id
text stem
mcq_instruction_type instruction
jsonb options
nextgen_subject_array foundational_concepts
}
```
Sources: [apps/web-svelte/database/schema.sql:356-400](), [apps/web-svelte/database/schema.sql:535-560]()
## Session Lifecycle and Management
The MCQ practice workflow moves through three primary phases: configuration, active testing, and results review.
### Session Initialization
Users configure their practice set by selecting subjects (e.g., Contracts, Torts, Evidence), difficulty levels, and a feedback mode. The system supports two feedback modes:
* **Review as I go**: Provides immediate feedback and explanations after each question.
* **Review at end**: Simulates exam conditions, withholding feedback until the entire set is submitted.
Sources: [apps/web-svelte/src/routes/nextgen/mcq/+page.svelte:20-25](), [apps/web-svelte/src/lib/exam-types/nextgen/mcq-types.ts:28-29]()
### Active Session Flow
```mermaid
sequenceDiagram
participant User as User Interface
participant State as ExamSessionState
participant API as Server API
participant DB as Database
User->>State: Select Option / Strikeout
State->>State: Update version number
Note right of State: Versioning prevents stale-save overwrites
loop Every 30 seconds
State->>API: PUT /api/sessions/[id]/save
API->>DB: Upsert mcq_responses (version check)
end
User->>State: Finish Session
State->>API: POST /nextgen/mcq/[sessionId]?/finish
API->>DB: Update status to COMPLETED
API-->>User: Redirect to Results
```
Sources: [apps/web-svelte/src/routes/nextgen/mcq/[sessionId]/+page.svelte:61-100](), [apps/web-svelte/src/routes/nextgen/mcq/[sessionId]/exam-session-state.svelte.ts:160-185]()
## Client-Side State Management
The `ExamSessionState` factory function utilizes Svelte 5 runes (`$state`, `$derived`) and reactive collections (`SvelteMap`, `SvelteSet`) to manage the session in memory.
### Key Functional Responsibilities
* **Navigation**: Tracking `currentIndex` and handling `goNext`/`goBack` operations.
* **Time Tracking**: Recording time spent per question in milliseconds to ensure rapid navigation doesn't result in lost data.
* **Versioning**: Incrementing a `responseVersion` for every answer change to support the server's stale-save protection (using `WHERE mcq_responses.version < EXCLUDED.version`).
* **Serialization**: Transforming reactive maps into a format suitable for JSON payloads during autosave or final submission.
Sources: [apps/web-svelte/src/routes/nextgen/mcq/[sessionId]/exam-session-state.svelte.ts:47-150]()
## Scoring and Results Analysis
Upon completion, sessions are graded server-side. The results interface calculates performance across several dimensions.
### Summary Metrics
The system computes an `McqSessionSummary` containing:
* **Overall Accuracy**: Total correct answers and partial credit (for select-two questions).
* **Subject Breakdown**: Performance metrics mapped to `NextgenSubject` enums.
* **Performance Tier**: Categorization into "Mastery", "Proficient", "Developing", or "Building Foundation" based on accuracy percentages.
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/mcq-types.ts:104-109](), [apps/web-svelte/src/lib/nextgen/components/exam/McqResults.svelte:34-55]()
### Response Projection
For security, the server projects `McqResponseRow` data into a public shape (`McqResponsePublic`) before sending it to the Results UI. This strips internal identifiers like `user_id` and `session_id`, only providing `question_id`, `selected_options`, `strikeouts`, and correctness flags.
Sources: [apps/web-svelte/src/routes/nextgen/mcq/[sessionId]/results/+page.server.ts:25-36](), [apps/web-svelte/src/lib/nextgen/components/exam/McqResults.svelte:10-17]()
## Summary
MCQ & Question Handling provides a robust framework for bar exam preparation by combining reactive frontend state with a versioned, persistent backend. The system's ability to handle complex instruction types (select-two), offer flexible feedback modes, and provide detailed subject-level analytics ensures a comprehensive practice experience for candidates preparing for the NextGen UBE.
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/mcq-types.ts](), [apps/web-svelte/src/routes/nextgen/mcq/[sessionId]/exam-session-state.svelte.ts]()
## Data Management/Flow
### Database Migrations & Schema
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/database/MIGRATIONS.md](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/MIGRATIONS.md)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [apps/web-svelte/scripts/migrate.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/migrate.ts)
- [apps/web-svelte/scripts/generate-schema-snapshot.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/generate-schema-snapshot.ts)
- [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql)
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
# Database Migrations & Schema
The database architecture for Bar Prep by SHEP is built on a PostgreSQL foundation (hosted via Neon) using a dual-database strategy. It manages user progress, exam content, and practice sessions for both NextGen UBE and MEE exam types. The system employs a rigorous migration-based workflow to ensure schema consistency across development, staging, and production environments.
The schema is divided into shared core tables (users, sessions, drafts), exam-specific modules (NextGen, MEE, MCQ), and an advanced V3.2 authoring and grading engine. This architecture supports high-fidelity practice simulations, including progressive disclosure for NextGen Integrated Question Sets (IQS) and detailed rubric-based grading.
Sources: [CLAUDE.md:46-51](), [apps/web-svelte/database/migrations/001_initial_schema.sql:1-15]()
## Migration Workflow & Tooling
The project uses a custom TypeScript-based migration runner rather than an off-the-shelf ORM migration tool. This ensures that the schema is defined in pure SQL while maintaining a historical record of changes.
### Lifecycle of a Migration
1. **Forward Migration:** A file named `NNN_title.sql` is created in `database/migrations/`.
2. **Rollback Script:** A companion `NNN_title.rollback.sql` is created for reversibility.
3. **Application:** The `npm run migrate` command executes pending forward migrations and records them in the `schema_migrations` ledger.
4. **Snapshotting:** The `npm run migrate:snapshot` command rebuilds a clean database from scratch and generates a canonical `schema.sql` snapshot.
Sources: [apps/web-svelte/database/MIGRATIONS.md:5-15](), [apps/web-svelte/scripts/migrate.ts:40-55]()
### Schema Snapshot Generation
The snapshot process uses a scratch database (or a local Docker container) to replay all migrations. This ensures `schema.sql` reflects a "clean" state free from manual database drift.
```mermaid
flowchart TD
Start[npm run migrate:snapshot] --> CheckDB{Scratch DB Configured?}
CheckDB -- No --> Docker[Launch Temporary Docker Postgres]
CheckDB -- Yes --> Reset[Reset Scratch DB Schema]
Docker --> Reset
Reset --> Apply[Apply All Forward Migrations]
Apply --> Dump[Execute pg_dump --schema-only]
Dump --> Normalize[Normalize Dump Output]
Normalize --> Write[Update apps/web-svelte/database/schema.sql]
```
The snapshotting tool automates the validation of the entire migration history.
Sources: [apps/web-svelte/scripts/generate-schema-snapshot.ts:133-157]()
## Core Schema Architecture
The database implements a "Shadow User" pattern to integrate with the SHEP core product.
### Dual-Database Integration
| Database URL | Purpose |
| :--- | :--- |
| `DATABASE_URL` | Primary barprep DB: contains exam data, sessions, tasks, drafts, and local `users`. |
| `AUTH_DATABASE_URL` | SHEP core DB: Auth.js writes here so barprep users exist in the master user table. |
Sources: [CLAUDE.md:46-56]()
### Entity Relationship Overview
The following diagram illustrates the relationship between shared user tables and exam-specific content.
```mermaid
erDiagram
USERS ||--o{ PRACTICE_SESSIONS : "starts"
USERS ||--o{ DRAFTS : "saves"
USERS ||--o{ PAYMENTS : "makes"
PRACTICE_SESSIONS ||--o| NEXTGEN_TASKS : "references"
PRACTICE_SESSIONS ||--o| MEE_ESSAYS : "references"
PRACTICE_SESSIONS ||--o{ MCQ_RESPONSES : "contains"
NEXTGEN_TASKS ||--o{ NEXTGEN_TASK_MATERIALS : "contains"
NEXTGEN_LEGAL_MATERIALS ||--o{ NEXTGEN_TASK_MATERIALS : "assigned_to"
MEE_ESSAYS ||--o{ MEE_MODEL_ANSWERS : "has"
PRACTICE_SESSIONS ||--o| NEXTGEN_SUBMISSIONS : "finalized_as"
PRACTICE_SESSIONS ||--o| MEE_SUBMISSIONS : "finalized_as"
```
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:35-130](), [apps/web-svelte/database/migrations/003_mcq_tables.sql:40-65]()
## Domain-Specific Data Models
### Shared Session Management
The `practice_sessions` and `drafts` tables use polymorphic references to handle different exam types within single tables, enforced by `CHECK` constraints.
```sql
CONSTRAINT valid_task_reference CHECK (
(exam_type = 'NEXTGEN' AND nextgen_task_id IS NOT NULL AND mee_essay_id IS NULL) OR
(exam_type = 'MEE' AND mee_essay_id IS NOT NULL AND nextgen_task_id IS NULL)
)
```
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:68-71](), [apps/web-svelte/database/schema.sql:475-479]()
### NextGen Content Model
NextGen tasks focus on legal research and performance. The schema supports a library of legal materials linked to specific tasks via a junction table that includes a `disclosure_step` for progressive reveal.
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:106-115]()
### MEE Content Model
MEE essays focus on recall. The schema includes support for fact patterns, "call of the question" prompts, and structured model answers containing IRAC (Issue, Rule, Application, Conclusion) breakdowns in JSONB format.
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:135-155]()
### MCQ Engine
The MCQ module supports multiple instruction types (`select_one`, `select_two`) and tracks detailed response data, including "strikeouts" (eliminated options) and time spent per question.
Sources: [apps/web-svelte/database/migrations/003_mcq_tables.sql:40-58]()
## V3.2 Authoring & Grading Engine
Migration 006 introduced a comprehensive set of tables to support the V3.2 grading engine, which utilizes LLMs for evaluation and certification.
### Canonical Versioned Artifacts
The system maintains strict versioning for evaluation components:
* **`packet_versions`**: Authoritative exam content payloads.
* **`rubric_versions`**: Grading criteria, including dimensions like spotting and analysis.
* **`bundle_versions`**: Compiled runtime artifacts linking a packet, rubric, and benchmark suite.
* **`model_manifests`**: Configuration for specific LLM models, including prompt versions and data retention modes.
Sources: [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql:100-160]()
### Quality & Certification
To ensure evaluation accuracy, the V3.2 schema includes automated reporting tables:
* **`self_consistency_reports`**: Records multiple LLM passes to ensure stable grading.
* **`certification_reports`**: Validates grading engine performance against benchmark suites.
* **`retention_posture_checks`**: Verifies that upstream LLM providers adhere to requested data privacy settings (e.g., zero-retention).
Sources: [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql:180-235](), [apps/web-svelte/database/schema.sql:1100-1115]()
## Conclusion
The database schema of Bar Prep by SHEP is a sophisticated multi-domain system that balances the simplicity of shared session tracking with the complexity of an LLM-powered grading engine. By using a strict migration and snapshotting workflow, the project maintains high technical standards for data integrity and deployment reliability. This architecture provides the necessary infrastructure to support the transition from traditional MEE essays to the skills-based NextGen Bar Exam.
### Practice Session Management
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/sessions.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/sessions.ts)
- [apps/web-svelte/src/routes/api/sessions/[id]/save/+server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/api/sessions/%5Bid%5D/save/%2Bserver.ts)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [apps/web-svelte/src/lib/shared/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/shared/types.ts)
- [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/%2Bpage.svelte)
- [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/%2Bpage.server.ts)
- [apps/web-svelte/src/routes/mee/practice/[essayId]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/%5BessayId%5D/%2Bpage.server.ts)
# Practice Session Management
The Practice Session Management system is responsible for orchestrating the lifecycle of a user's engagement with bar exam preparation materials, specifically targeting the **NextGen UBE** and **MEE** (Multistate Essay Examination) modules. It serves as the bridge between the user interface and the persistent storage layer, ensuring that progress, timing, and draft content are accurately tracked and secured throughout a practice attempt.
The system handles polymorphic session types, allowing a unified management interface for different exam formats while maintaining strict data integrity through stale-save guards and HTML sanitization. It supports features such as real-time timers, automated background saving, and the ability to resume interrupted sessions.
## Session Lifecycle and States
Every practice attempt is governed by a state machine that transitions based on user actions or system triggers (e.g., timeouts). Sessions are tracked using the `practice_session_status` enum.
```mermaid
stateDiagram-v2
[*] --> IN_PROGRESS : User starts task
IN_PROGRESS --> COMPLETED : User submits analysis
IN_PROGRESS --> ABANDONED : User discards work
IN_PROGRESS --> TIMED_OUT : Timer expires (System)
COMPLETED --> [*]
ABANDONED --> [*]
TIMED_OUT --> COMPLETED : User submits after timeout
```
The diagram shows the possible transitions for a practice session, where `IN_PROGRESS` is the primary active state for data collection.
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:28](), [apps/web-svelte/src/lib/shared/types.ts:11]()
### State Descriptions
| Status | Description |
| :--- | :--- |
| `IN_PROGRESS` | The default state when a session is created. Permits updates to elapsed time and content. |
| `COMPLETED` | Finalized state set upon submission. The session is no longer editable. |
| `ABANDONED` | The user has explicitly chosen to discard the session results. |
| `TIMED_OUT` | Triggered when the allocated exam time has elapsed, though users may still review and submit. |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:28](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:315]()
## Data Architecture
The system utilizes a polymorphic database schema to handle different task types (NextGen tasks vs. MEE essays) within a single `practice_sessions` table.
### Database Schema
The core table `practice_sessions` links users to specific tasks and tracks session metadata.
| Field | Type | Description |
| :--- | :--- | :--- |
| `id` | UUID | Primary identifier for the session. |
| `user_id` | UUID | Foreign key to the local shadow users table. |
| `exam_type` | `exam_type` | Enum: `NEXTGEN` or `MEE`. |
| `nextgen_task_id` | UUID | Reference to NextGen task (NULL for MEE). |
| `mee_essay_id` | UUID | Reference to MEE essay (NULL for NextGen). |
| `elapsed_seconds` | INT | Total time spent by the user in the session. |
| `resume_context` | JSONB | Stores auxiliary state like notepad content and marked flags. |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:67-91](), [apps/web-svelte/src/lib/server/sessions.ts:108-142]()
### Relationship Diagram
```mermaid
erDiagram
USER ||--o{ PRACTICE_SESSION : "owns"
PRACTICE_SESSION ||--o| NEXTGEN_TASK : "references"
PRACTICE_SESSION ||--o| MEE_ESSAY : "references"
PRACTICE_SESSION ||--o| DRAFT : "persists"
PRACTICE_SESSION ||--o| SUBMISSION : "finalizes"
USER {
uuid id
enum exam_type
}
PRACTICE_SESSION {
uuid id
enum status
int elapsed_seconds
jsonb resume_context
}
```
This ER diagram illustrates how sessions act as the central node connecting users, content (Tasks/Essays), and their work products (Drafts/Submissions).
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:67-123]()
## Persistence and Synchronization
The persistence layer ensures that user progress is saved reliably without manual intervention, while protecting against concurrent or stale writes.
### Auto-save Mechanism
On the client side, a Svelte `$effect` hook triggers a background save every 30 seconds while the session is `IN_PROGRESS`. This updates the server with the current writing content and elapsed time.
```mermaid
sequenceDiagram
participant C as Client (Svelte)
participant A as API (+server.ts)
participant D as Database (Postgres)
C->>C: Interval (30s)
C->>A: PUT /api/sessions/[id]/save
Note right of A: Validate sessionId & status
A->>D: SELECT updated_at FROM drafts
alt Save is Stale
D-->>A: Newer updated_at exists
A-->>C: 409 Conflict (Stale rejected)
else Save is Fresh
A->>D: BEGIN Transaction
A->>D: UPDATE practice_sessions (elapsed)
A->>D: UPSERT drafts (content)
A->>D: COMMIT
D-->>A: Success
A-->>C: 200 OK (savedAt)
end
```
The sequence above details the stale-save guard logic implemented at the persistence boundary to prevent late-arriving network requests from overwriting newer content.
Sources: [apps/web-svelte/src/routes/api/sessions/[id]/save/+server.ts:47-92](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:61-82]()
### Key Functions and API Endpoints
* **`createSession`**: Initializes a new session record in the database. For MCQ sets, it also stores the `mcq_set_config`.
* *Path:* `src/lib/server/sessions.ts:4-28`
* **`saveRequestSchema`**: A Zod schema used to validate incoming save requests for both written assignments and MCQ responses.
* *Path:* `src/routes/api/sessions/[id]/save/+server.ts:18`
* **`sanitizeDraftHtml`**: Strips malicious markup from user-supplied HTML before persistence to prevent stored XSS.
* *Path:* `src/routes/api/sessions/[id]/save/+server.ts:44`
## Implementation Details
### NextGen vs. MEE Initialization
The server-side loaders (`+page.server.ts`) for both NextGen and MEE follow a pattern of checking for an existing `activeSession` before allowing a new one to be created via the `start` action.
```typescript
// Example from NextGen session start
export const actions: Actions = {
start: async (event) => {
const barprepUserId = requireBarprepUserId(session);
const taskId = event.params.taskId;
const existing = await getActiveSession(barprepUserId, { nextgenTaskId: taskId });
if (existing) {
return { sessionId: existing.id };
}
// ... create new session logic ...
}
};
```
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts:36-61](), [apps/web-svelte/src/routes/mee/practice/[essayId]/+page.server.ts:33-55]()
### Stale-Save Guard Logic
The `PUT` handler in the save API uses an atomic transaction to compare the `savedAt` timestamp provided by the client with the `updated_at` timestamp in the database. If the client’s timestamp is older, the request is rejected with a `409 Conflict`.
Sources: [apps/web-svelte/src/routes/api/sessions/[id]/save/+server.ts:56-65]()
## Conclusion
Practice Session Management is a critical system that ensures a seamless and secure experience for bar exam candidates. By leveraging polymorphic database relations, robust server-side validation, and background synchronization, it maintains the integrity of high-stakes practice data across diverse exam formats and device types. Summary: [apps/web-svelte/src/lib/server/sessions.ts](), [apps/web-svelte/src/routes/api/sessions/[id]/save/+server.ts]()
### Content Pipeline & Packet Structure
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts)
- [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [apps/web-svelte/scripts/seed-all-packets.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/seed-all-packets.ts)
# Content Pipeline & Packet Structure
The Bar Prep by SHEP content pipeline is a multi-phase system designed to ingest, compile, and validate legal practice materials for the NextGen Bar Exam and the Multistate Essay Examination (MEE). The system transforms high-level "Canonical Packets"—which contain the legal fact patterns, issues, and rules—into "Executable Scoring Bundles" used by the AI-powered grading engine to evaluate student submissions.
This architecture ensures that practice sessions are grounded in structured legal doctrine while allowing for automated, high-fidelity scoring against complex multi-dimensional rubrics. The pipeline supports various content tiers, including licensed NCBE materials and SHEP-authored practice tasks. Sources: [PRODUCT_SPEC.md:104-121](), [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:6-14]()
## Canonical Packet Structure
The `CanonicalPacket` is the primary authoring format for MEE essays. It serves as a declarative blueprint containing the prompt, the underlying legal doctrine, and the expected analytical paths. Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:145-200]()
### Key Schema Components
| Component | Description |
| :--- | :--- |
| **Prompt** | Contains the raw `prompt_text`, specific `calls` (questions), and a list of `facts` with salience levels (core, supporting, distractor). |
| **Issues** | A collection of legal issues. Each issue includes `recognition_aliases`, `rule_atoms`, `fact_refs`, and `acceptable_paths`. |
| **Acceptable Paths** | Logic structures defining how rule atoms, facts, and reasoning moves must be combined to reach valid conclusions. |
| **Whole Essay Modifiers** | Flags for cross-cutting metrics like responsiveness, coherence, and incompleteness. |
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:25-45](), [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:77-105]()
```mermaid
classDiagram
class CanonicalPacket {
+string packet_id
+string title
+Prompt prompt
+Issue[] issues
+WholeEssayModifiers modifiers
}
class Prompt {
+string prompt_text
+Call[] calls
+Fact[] facts
}
class Issue {
+string issue_id
+string label
+RuleAtom[] rule_atoms
+AcceptablePath[] acceptable_paths
+Caps caps
}
class AcceptablePath {
+string[] required_rule_atom_ids
+string[] required_fact_refs
+string[] acceptable_conclusions
}
CanonicalPacket *-- Prompt
CanonicalPacket *-- Issue
Issue *-- AcceptablePath
```
The diagram above illustrates the hierarchical relationship between the core packet metadata and the nested legal logic components. Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:145-230]()
## Compilation Pipeline
The compiler transforms a `CanonicalPacket` into an `ExecutableScoringBundle`. This process involves several distinct phases that normalize data and derive the scoring logic required for the runtime engine. Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:1-5]()
### Phase 1: Normalization and Derivation
* **normalizePacket**: Deep-clones the packet and sorts arrays (e.g., aliases, rule atoms) to ensure deterministic output for hashing. Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:11-19]()
* **deriveIssueBundles**: Generates checklist items for each issue across five dimensions: `spotting`, `rule`, `facts`, `analysis`, and `conclusion`. Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:88-106]()
### Phase 2: Policy Emission
* **emitWholeEssayBundle**: Filters and configures metrics for responsiveness, coherence, and irrelevance based on packet modifiers. Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:148-164]()
* **emitAggregationPolicy**: Configures how individual issue scores are combined into a final band (1-6), including threshold validation. Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:192-205]()
* **emitHardBandRules**: Generates deterministic rules that can cap a student's score, such as "blank response" or "missing top-band requirement." Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:213-254]()
```mermaid
flowchart TD
Start[Canonical Packet] --> Norm[Normalize Packet]
Norm --> Issues[Derive Issue Bundles]
Issues --> Policies[Emit Policies & Modifiers]
Policies --> HardRules[Emit Hard Band Rules]
HardRules --> Bundle[Executable Scoring Bundle]
subgraph Dimension Checklist
Issues --> D1[Spotting]
Issues --> D2[Rule]
Issues --> D3[Facts]
Issues --> D4[Analysis]
Issues --> D5[Conclusion]
end
```
This flowchart shows the transformation flow from a source packet to the final executable bundle. Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:1-260]()
## Content Persistence and Seeding
The project uses a migration-based SQL schema to manage content across NextGen and MEE domains. Packets and their compiled versions are stored in dedicated tables to support versioning and runtime access. Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:1-10]()
### Database Tables
| Table | Purpose |
| :--- | :--- |
| `v3_2_packet_versions` | Stores the source JSON for canonical packets and their lifecycle status (draft, published, etc.). |
| `v3_2_bundle_versions` | Stores compiled `ExecutableScoringBundle` JSON and references to the source packet/rubric. |
| `nextgen_tasks` | Metadata for NextGen scenarios, including task type (SPT, LRPT, IQS) and supervisor memos. |
| `nextgen_legal_materials` | The library of legal sources (statutes, cases, restatements) available to students during NextGen tasks. |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:98-130](), [apps/web-svelte/database/schema.sql:845-865]()
### Seeding Logic
The system includes specialized scripts (`seed-all-packets.ts`) that use the production compiler to generate identical bundles for development and testing environments. This ensures consistency between authored content and the live database state. Sources: [apps/web-svelte/scripts/seed-all-packets.ts:12-25]()
## Benchmark and Validation
To ensure the technical accuracy of the content pipeline, every packet is paired with a `BenchmarkSuite`. This suite contains "anchors" (exemplar student responses) that have been manually or AI-graded to establish the expected scoring behavior of the bundle. Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:160-180]()
### Validation Requirements
* **Weight Sum**: Total weight of all issues in a packet must sum to exactly 1.0 (±0.01). Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:223-230]()
* **Band Exemplars**: Depending on the `schema_profile` (e.g., `llm_only_launch`), packets must include specific band anchors (typically at bands 2, 4, and 6) to calibrate the grading engine. Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:205-215]()
* **Threshold Contiguity**: Band thresholds must be contiguous, start at 0, and end at 1 without gaps or overlaps. Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:172-190]()
## Conclusion
The Content Pipeline & Packet Structure forms the backbone of the Bar Prep by SHEP platform. By separating the authoring of legal doctrine (`CanonicalPacket`) from the technical scoring configuration (`ExecutableScoringBundle`), the system achieves a balance between legal pedagogical quality and automated grading precision. The use of strict Zod schemas and a multi-phase compilation pipeline ensures that every practice task is verified and calibrated before reaching students. Sources: [PRODUCT_SPEC.md:123-135](), [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts:275-300]()
## Frontend Components
### SHEP Design System & Theming
Relevant source files
The following files were used as context for generating this wiki page:
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [apps/web-svelte/src/app.css](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/app.css)
- [apps/web-svelte/src/app.html](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/app.html)
- [build_app_css.py](https://github.com/airobal/barprep-by-shep/blob/main/build_app_css.py)
- [apps/web-svelte/src/lib/stores/theme.svelte.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/stores/theme.svelte.ts)
- [apps/web-svelte/src/lib/stores/device.svelte.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/stores/device.svelte.ts)
# SHEP Design System & Theming
The SHEP Design System is a comprehensive UI framework ported from the core SHEP product to the Bar Prep application. It focuses on a serious, scholarly aesthetic tailored for legal reasoning practice, utilizing semantic design tokens, high-fidelity typography, and a sophisticated OKLCH-based color system. The system provides a unified visual language across the **NextGen UBE** and **MEE** exam modules while maintaining a separate product identity.
Sources: [HANDOFF.md:12-14](), [PRODUCT_SPEC.md:173-177](), [CLAUDE.md:68-69]()
## Visual Foundation & Architecture
The design system is built on Tailwind CSS 4 and Svelte 5, using CSS custom properties for semantic mapping. The architecture prioritizes "elevation" via lightness deltas and consistent spacing scales.
### Color System (OKLCH)
The system uses the OKLCH color space, which allows for perceptually uniform color manipulation. A key architectural principle is the use of **8% OKLCH lightness deltas** to define the surface hierarchy:
* **Body**: Base layer (`--bg-body`)
* **Card**: Elevated surface (`--bg-card`)
* **Elevated**: Interaction/High-focus surface
* **Overlay**: Modal and menu layers
Sources: [HANDOFF.md:73-74](), [CLAUDE.md:71-73]()
### Typography Stack
The system employs a specific set of typefaces to evoke a "Light Academia" and professional legal atmosphere:
| Font Family | Usage | Mapping |
|:---|:---|:---|
| **Cormorant Garamond** | Display, Headings | `font-display` |
| **Source Serif 4** | Body Prose, Fact Patterns | `font-body` |
| **Geist Sans** | UI elements, navigation | `font-sans` |
| **Cinzel** | Labels, uppercase metadata | `font-label` |
| **Geist Mono** | Code, timers, tabular data | `font-mono` |
Sources: [CLAUDE.md:74-76](), [apps/web-svelte/src/app.html:10-18]()
## Theme Management
The theming engine supports multiple presets and prevents Flash of Unstyled Content (FOUC) through a specialized initialization flow.
### Theme Presets
The system defines five primary theme variants, categorized by their light/dark modes:
1. **Light**: Standard academia-inspired light mode.
2. **Light Paper**: Tactile, high-readability light mode with paper textures.
3. **Dark**: Standard dark mode.
4. **Dark Paper**: Muted dark mode with texture.
5. **Night**: OLED-optimized deep dark mode.
Sources: [CLAUDE.md:71](), [apps/web-svelte/src/app.html:28-30]()
### Initialization Flow
To prevent FOUC, the system executes an inline script in `app.html` to apply classes before the first paint. It then synchronizes this with a Svelte 5 `$state` store for reactive updates.
```mermaid
flowchart TD
A[Browser Load app.html] --> B{Check localStorage}
B -- theme exists --> C[Apply .dark / data-theme]
B -- no theme --> D[Check prefers-color-scheme]
D --> C
C --> E[First Paint]
E --> F[Initialize theme.svelte.ts]
F --> G[Reactivate State Store]
```
The `theme.svelte.ts` store manages the persistent state of both `mode` (light/dark) and `variant` (classic/paper/night).
Sources: [apps/web-svelte/src/app.html:21-42](), [apps/web-svelte/src/lib/stores/theme.svelte.ts]()
## Design Tokens and Implementation
Tokens are centralized in `apps/web-svelte/src/app.css`. These are processed from the SHEP core via a Python script (`build_app_css.py`) that strips event-specific and workflow-specific tokens irrelevant to the Bar Prep product.
### Spacing and Motion
* **Spacing Scale**: Defined as `--space-1` through `--space-12`, exposed via Tailwind as `ds-1` through `ds-12`.
* **Motion Tokens**:
* `--duration-fast`: 120ms
* `--duration-normal`: 200ms
* `--duration-slow`: 320ms
* **Accessibility**: All motion respects the `prefers-reduced-motion` media query.
Sources: [CLAUDE.md:77-80](), [build_app_css.py:1-10]()
### Component Hierarchy
The design system provides a suite of ported components that inherit these tokens:
```mermaid
classDiagram
class DesignTokens {
+OKLCH Color Palette
+Typography Scale
+Motion Durations
+Z-Index Scale
}
class BaseStyles {
+CSS Custom Properties
+Semantic Mappings
}
class UIComponents {
+Card (variants: classic, paper)
+Button (variants: primary, ghost, danger)
+Badge
+Tabs
}
DesignTokens <|-- BaseStyles
BaseStyles <|-- UIComponents
```
Sources: [HANDOFF.md:20-30](), [CLAUDE.md:82-86]()
## Responsive Design (Device Store)
The `device.svelte.ts` store utilizes Svelte 5 runes to provide reactive breakpoints. It includes hysteresis to prevent rapid switching between mobile and desktop states at the breakpoint boundaries.
### Breakpoint Constants
* **Mobile**: `< 768px`
* **Tablet**: `768px - 1024px`
* **Desktop**: `> 1024px`
Sources: [HANDOFF.md:18](), [CLAUDE.md:57](), [apps/web-svelte/src/lib/stores/device.svelte.ts]()
## Summary
The SHEP Design System & Theming module provides a robust, professional-grade visual infrastructure. By utilizing OKLCH lightness deltas for depth, high-fidelity serif typography for readability, and a resilient initialization strategy, it ensures a consistent and scholarly environment for bar exam preparation. The system is designed for isolation, with a dedicated build pipeline to maintain parity with the core product while excluding extraneous features.
Sources: [HANDOFF.md:12-14](), [PRODUCT_SPEC.md:173-177](), [CLAUDE.md:71-73]()
### Core UI Components
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/components/Card.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/components/Card.svelte)
- [apps/web-svelte/src/lib/components/Button.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/components/Button.svelte)
- [apps/web-svelte/src/lib/components/Badge.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/components/Badge.svelte)
- [apps/web-svelte/src/lib/components/exam/RichTextEditor.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/components/exam/RichTextEditor.svelte)
- [apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte)
- [apps/web-svelte/src/app.css](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/app.css)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
# Core UI Components
Core UI components in the Bar Prep by SHEP project are built using **Svelte 5** and **Tailwind CSS 4**. They are ported directly from the SHEP core design system, utilizing a shared set of design tokens for typography, OKLCH color palettes, and motion effects. These components form the building blocks for the application's practice interfaces, dashboards, and landing pages, ensuring a consistent "Superhuman-style" aesthetic and professional legal reasoning environment.
The system emphasizes high-performance interactions, using Svelte 5 runes (`$state`, `$derived`, `$props`) for reactive state management and CSS custom properties for theme-aware styling.
Sources: [HANDOFF.md:16-22](), [CLAUDE.md:83-88]()
## Design Token System
The foundational styling of all UI components is driven by a centralized design token system defined in `app.css`. This system uses the **OKLCH** color space to maintain consistent lightness deltas across themes (Light, Dark, Paper, and Night).
### Color and Elevation
The project employs an 8% OKLCH lightness delta strategy for elevation. This creates a clear visual hierarchy between the page body, cards, elevated surfaces, and overlays.
| Level | Light L% | Dark L% | Night L% | Description |
|-------|----------|---------|----------|-------------|
| Body | 97% | 15% | 7% | Main background surface |
| Card | 99% | 23% | 12% | Standard content containers |
| Elevated | 99% | 31% | 22% | Surfaces lifted above cards |
| Overlay | 99% | 39% | 30% | Modals and popovers |
Sources: [app.css:124-135](), [HANDOFF.md:73-75]()
### Typography
The system uses a 5-tier font stack to balance display elegance with technical readability.
* **Display:** Cormorant Garamond (Headings)
* **Body:** Source Serif 4 (Legal prose)
* **Sans/UI:** Geist Variable (Navigation and controls)
* **Label:** Cinzel (Headers and uppercase labels)
* **Mono:** Geist Mono (Technical data)
Sources: [app.css:218-228](), [HANDOFF.md:71-72]()
## Layout and Structural Components
### Card Component
The `Card` is the primary container for grouping related content. It supports multiple variants (default, bordered, elevated, flat) and features specialized visual effects like "grain" (tactile paper texture) and "interactive glow."
```mermaid
graph TD
Props[Component Props] --> Logic{Interactive?}
Logic -- Yes --> Anchor[Render 'a' Tag]
Logic -- No --> Div[Render 'div' Tag]
Anchor --> Slots[Render Snippets]
Div --> Slots
Slots --> Header[Header Snippet]
Slots --> Children[Main Content]
Slots --> Footer[Footer Snippet]
style Anchor fill:#f9f,stroke:#333
style Div fill:#bbf,stroke:#333
```
*Figure 1: Decision logic for Card component rendering based on interactivity props.*
Sources: [apps/web-svelte/src/lib/components/Card.svelte:1-35](), [app.css:864-890]()
### Exam Shell and Headers
The `ExamShell` provides the high-level application layout for practice sessions, coordinating the header, navigation bar, sidebar, and footer. The `ExamHeader` manages global session state, such as the timer and platform settings.
Sources: [apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte:1-40](), [apps/web-svelte/src/lib/components/exam/ExamHeader.svelte:1-25]()
## Input and Interactive Components
### Rich Text Editor
A core requirement for legal practice is a robust writing environment. The `RichTextEditor` is a `contenteditable` surface that handles sanitized HTML input and provides a formatting toolbar for legal analysis (Bold, Italic, Headings, Lists, Blockquotes).
**Key Editor Logic:**
1. **Sanitization:** Uses `isomorphic-dompurify` to strip unsafe tags on initial load.
2. **Command Execution:** Wraps `document.execCommand` for cross-browser formatting.
3. **State Tracking:** Tracks active formatting (e.g., is current selection bold?) to update the toolbar UI.
4. **Paste Handling:** Intercepts paste events to strip external formatting and force plain text.
Sources: [apps/web-svelte/src/lib/components/exam/RichTextEditor.svelte:1-110]()
### Button and Badge
Buttons and Badges utilize semantic variant mappings to ensure theme compatibility.
| Component | Variants | Usage |
|-----------|----------|-------|
| **Button** | primary, secondary, outline, ghost, danger | Action triggers with support for loading states and icons. |
| **Badge** | success, warning, error, info, purple | Status indicators or metadata tags. |
Sources: [apps/web-svelte/src/lib/components/Button.svelte:1-20](), [apps/web-svelte/src/lib/components/Badge.svelte:1-15]()
## Interactive Systems
### Love Interaction
The "Love Interaction" system is a proximity-driven animation framework. Elements respond to cursor approach before an actual hover event occurs.
```mermaid
sequenceDiagram
participant Mouse as User Cursor
participant Action as loveInteraction Action
participant CSS as app.css Variable
participant UI as Component
Mouse->>Action: Cursor moves within radius
Action->>CSS: Update --love-proximity (0 to 1)
CSS->>UI: Apply filter: brightness()
CSS->>UI: Apply transform: translateY()
CSS->>UI: Transition box-shadow
Note over UI: Visual "glow" intensifies as cursor nears
```
*Figure 2: Data flow for proximity-driven "Love" interactions.*
Sources: [app.css:986-1025](), [apps/web-svelte/src/routes/nextgen/+page.svelte:137-145]()
### Motion and Transitions
Components utilize standardized motion tokens for consistent timing.
* **Fast (`120ms`):** Hover states and toggle switches.
* **Normal (`200ms`):** Sidebar expansions and dropdowns.
* **Slow (`320ms`):** Page content reveals and modal entries.
Sources: [app.css:243-251](), [HANDOFF.md:77-79]()
Core UI components prioritize technical accuracy and legal professionalism. By leveraging the SHEP design system's OKLCH color scaling and specialized typography, the components provide an environment that mimics official NCBE interfaces while maintaining modern web performance standards through Svelte 5.
### NextGen Specific Components
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte)
- [apps/web-svelte/src/lib/nextgen/components/AssignmentMemo.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/AssignmentMemo.svelte)
- [apps/web-svelte/src/lib/nextgen/components/MaterialsPanel.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/MaterialsPanel.svelte)
- [apps/web-svelte/src/lib/nextgen/components/PracticeTimer.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/components/PracticeTimer.svelte)
- [apps/web-svelte/src/lib/exam-types/nextgen/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/nextgen/types.ts)
- [apps/web-svelte/src/lib/nextgen/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/nextgen/types.ts)
- [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/nextgen/practice/%5BtaskId%5D/+page.svelte)
# NextGen Specific Components
The NextGen Specific Components comprise the specialized UI layer and data structures designed to replicate the National Conference of Bar Examiners (NCBE) NextGen Bar Exam environment. This system facilitates skills-based legal testing through structured performance tasks (SPT), legal research tasks (LRPT), and integrated question sets (IQS).
These components focus on a "split-pane" architectural pattern, professional legal document presentation, and progressive disclosure of materials. The system ensures that candidates interact with simulated law firm environments, including supervisor memos, client files, and legal libraries.
Sources: [HANDOFF.md](), [PRODUCT_SPEC.md](), [apps/web-svelte/src/lib/exam-types/nextgen/types.ts:1-10]()
## Core Component Architecture
The NextGen interface is built around several high-level functional areas: the Exam Shell, the Practice Editor (The Chambers), and the Materials System.
### The Exam Shell
The `ExamShell` serves as the primary layout wrapper for NextGen exam sessions. It manages the global state for the timer, navigation bar, and auxiliary tools like the notepad.
```mermaid
graph TD
Shell[ExamShell.svelte] --> Header[ExamHeader]
Shell --> Nav[ExamNavBar]
Shell --> Body[Main Content Area]
Shell --> Footer[ExamFooter]
Header --> Timer[ExamTimer.svelte]
Body --> Sidebar[ExamSidebar]
Body --> Content[{@render content}]
Shell -.-> Notepad[ExamNotepad Overlay]
```
The shell uses Svelte 5 runes (`$state`, `$derived`, `$effect`) to handle question transitions and UI persistence. It includes a "fade" effect during question navigation to preserve scroll positions while providing visual feedback.
Sources: [apps/web-svelte/src/lib/nextgen/components/exam/ExamShell.svelte:15-110]()
### The Chambers (Practice Editor)
The primary writing interface for Performance Tasks (SPT/LRPT) is found in the practice task route. It utilizes a draggable split-pane layout to allow simultaneous viewing of research materials and a rich-text writing area.
| Feature | Description | Implementation Detail |
| :--- | :--- | :--- |
| **Split-Pane Layout** | Draggable divider between materials and editor. | `practiceState.panelWidth` with pointer events. |
| **Focus Mode** | Hides materials to maximize writing space. | `practiceState.focusMode` toggle. |
| **Mobile Adaptation** | Switches from split-pane to tabbed navigation. | `device.isDesktop` check with `mobileTab` state. |
| **Auto-save** | Periodic background persistence of drafts. | 30-second `setInterval` targeting `/api/sessions/[id]/save`. |
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:60-150](), [HANDOFF.md]()
## Specialized Legal Components
### Assignment Memo
The `AssignmentMemo` component acts as the "entry point" for a task. It presents a professional law firm memorandum from a supervisor to the associate (user), outlining the specific legal assignment.
```mermaid
sequenceDiagram
participant U as User
participant AM as AssignmentMemo
participant S as Server
U->>AM: Views Task Summary
AM->>AM: Displays Supervisor Details & Instructions
AM->>AM: Lists Attached Documents
U->>AM: Clicks "Start Practice"
AM->>S: POST /start (Session Creation)
```
Sources: [apps/web-svelte/src/lib/nextgen/components/AssignmentMemo.svelte](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts:40-55]()
### Materials Panel
The `MaterialsPanel` manages the "Case File" and "Library" tabs. It uses `isomorphic-dompurify` to safely render legal content and authorities.
- **File Tab**: Contains the Statement of Facts, Background, and Client Document lists derived from the `SupervisorMemo` object.
- **Library Tab**: Contains legal authorities (Statutes, Cases, Regulations) sorted by `sort_order`.
Sources: [apps/web-svelte/src/lib/nextgen/components/MaterialsPanel.svelte:40-100]()
## Data Models and Type Definitions
The NextGen system relies on specific TypeScript interfaces to maintain consistency between the backend schema and the UI components.
### NextGen Task Types
Tasks are categorized into three primary formats, each with different UI requirements:
1. **SPT (Standard Performance Task)**: Extended writing based on a memo.
2. **LRPT (Legal Research Performance Task)**: Research-heavy drafting.
3. **IQS (Integrated Question Set)**: Progressive disclosure of information.
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/types.ts:1-20]()
### Key Interfaces
```typescript
export interface NextGenTask {
id: string;
taskType: NextGenTaskType;
rolePrompt: string;
supervisorMemo: SupervisorMemo | null;
writingAssignment: WritingAssignment | null;
}
export interface LegalMaterial {
id: string;
materialType: 'STATUTE' | 'CASE_EXCERPT' | 'REGULATION' | 'RESTATEMENT' | 'CLIENT_DOCUMENT';
content: string;
citation: string | null;
}
```
Sources: [apps/web-svelte/src/lib/exam-types/nextgen/types.ts:25-70](), [apps/web-svelte/src/lib/nextgen/types.ts:10-50]()
## Session and State Management
NextGen components utilize a factory pattern for instance-specific state management, particularly for the complex interactions in the practice editor.
```mermaid
flowchart TD
A[Page Load] --> B{Existing Session?}
B -- Yes --> C[Load Draft & Elapsed Time]
B -- No --> D[Show AssignmentMemo]
D -- Start --> E[Initialize createPracticeState]
C --> E
E --> F[Monitor Auto-save Timer]
F --> G[Submit Analysis]
G --> H[Final Sanitize & DB Save]
```
- **`createPracticeState`**: A factory function that manages panel widths, focus mode, content drafts, and mobile tab state using Svelte 5 runes.
- **Persistence**: Drafts are sanitized using `sanitizeDraftHtml` before being stored in the database to prevent XSS while allowing basic formatting.
Sources: [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.svelte:40-80](), [apps/web-svelte/src/routes/nextgen/practice/[taskId]/+page.server.ts:70-90]()
## Component Summary Paragraph
The NextGen Specific Components provide the essential infrastructure for the Bar Prep by SHEP application to deliver a high-fidelity simulation of the 2026 UBE. By combining structured data models for legal materials with highly interactive UI components like the `ExamShell` and `MaterialsPanel`, the system successfully bridges the gap between static legal content and dynamic exam practice. The use of Svelte 5 runes and a specialized state management pattern ensures that complex UI states, such as the split-pane editor and multi-step IQS flows, remain performant and responsive across all device types.
### MEE Specific Components
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/exam-types/mee/constants.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/mee/constants.ts)
- [apps/web-svelte/src/lib/exam-types/mee/types.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/exam-types/mee/types.ts)
- [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.svelte](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/packet/%5BpacketId%5D/+page.svelte)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
# MEE Specific Components
The Multistate Essay Examination (MEE) module provides a specialized environment for traditional essay-based bar exam preparation. Unlike the NextGen module which provides legal libraries, the MEE module focuses on pure recall, issue spotting, and the IRAC (Issue, Rule, Analysis, Conclusion) structure. The system encompasses specialized UI components for timed writing, complex data schemas for "Packets" and "Bundles," and an automated grading runtime.
Sources: [PRODUCT_SPEC.md](), [apps/web-svelte/src/lib/exam-types/mee/constants.ts:1-5]()
## Core Data Structures and Schemas
The MEE system relies on a tiered content structure consisting of Packets and Executable Scoring Bundles. These structures define the fact patterns, legal issues, and the scoring logic used by the AI grading engine.
### Canonical Packet
A `CanonicalPacket` is the source of truth for an essay question. It contains the prompt text, the specific "calls" (questions) to be answered, and the legal issues that a student is expected to identify.
| Field | Type | Description |
|-------|------|-------------|
| `packet_id` | `string` | Unique identifier for the essay packet. |
| `subject` | `MEESubject` | The primary legal subject (e.g., CIVIL_PROCEDURE). |
| `prompt` | `PromptSchema` | Contains the fact pattern and the calls of the question. |
| `issues` | `IssueSchema[]` | List of legal issues with associated rule atoms and fact refs. |
| `difficulty` | `enum` | Easy, Medium, Medium_Hard, or Hard. |
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:133-160]()
### Executable Scoring Bundle
The `ExecutableScoringBundle` is a compiled version of a packet used by the runtime engine. it includes aggregation policies, band thresholds (1–6), and hard band rules that can cap a student's score based on deterministic triggers like word count.
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts:258-295]()
## MEE UI Components and User Flow
The MEE practice interface is designed to transition from a "Pre-session" reading state to an "In-session" writing desk.
### The Writing Desk
The desk provides a distraction-free environment. On desktop, it features a split-pane view with the fact pattern on the left and a rich text editor on the right. On mobile, the prompt is collapsed into a disclosure component to maximize writing space.
```mermaid
flowchart TD
A[MEE Dashboard] --> B[Packet Landing Page]
B --> C{Timer Mode Selection}
C --> D[Begin Essay]
D --> E[Writing Desk]
E --> F[Rich Text Editor]
F --> G[Submit for Review]
G --> H[Grading Overlay]
H --> I[Results Page]
```
The flow from selection to submission. Sources: [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.svelte:145-250]()
### Components List
* **TimerModeSelector**: Allows users to choose between `timed_30_min` and `untimed` modes.
* **RichTextEditor**: A specialized editor for legal analysis.
* **Grading Overlay**: A modal that appears during submission, showing stages such as "Analyzing essay" and "Scoring against rubric."
* **IRAC Template**: An optional scaffold that users can insert to structure their response.
Sources: [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.svelte:45-120](), [apps/web-svelte/src/lib/exam-types/mee/constants.ts:54-65]()
## Grading Runtime Architecture
The MEE runtime handles the automated evaluation of submissions using a multi-channel LLM pipeline.
### Runtime Sequence
The `gradeSubmission` function executes the following steps:
1. **Preprocessing**: Segments raw text into sentences and paragraphs.
2. **Quality Channel**: An LLM call to assess whole-essay metrics like coherence and responsiveness.
3. **Retrieval**: Uses embeddings to find student text spans relevant to specific rubric issues.
4. **Packet Adherence**: Scores the student's response against the issue-specific checklist items.
5. **Evidence Verification**: A deterministic process that verifies if quoted text actually exists in the student's submission.
6. **Aggregation**: Combines scores and applies caps or modifiers to determine the final band (1–6).
```mermaid
sequenceDiagram
participant U as User
participant R as Runtime Engine
participant LLM as LLM Provider
participant DB as Database
U->>R: Submit Essay
R->>R: Preprocess Text
R->>LLM: Quality Channel Check
LLM-->>R: Coherence/Responsiveness Data
R->>LLM: Issue Adherence (Checklist)
LLM-->>R: Scoring + Evidence Quotes
R->>R: Verify Evidence (Deterministic)
R->>R: Aggregate Final Band (1-6)
R->>DB: Write Grading Run & Findings
R-->>U: Return Feedback & Score
```
The automated grading lifecycle. Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:58-150]()
## Database Schema (MEE Tables)
MEE-specific data is stored in tables that link back to shared user and session records.
| Table | Purpose |
|-------|---------|
| `mee_essays` | Stores the primary metadata, fact patterns, and difficulty levels for essays. |
| `mee_model_answers` | Contains licensed or professor-written model answers and IRAC breakdowns. |
| `mee_submissions` | Records student responses, word counts, and the scores/performance levels assigned by the runtime. |
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:133-176]()
### Subject Coverage
The system tracks 13 specific MEE subjects, categorized into "Core" subjects (e.g., Torts, Contracts) and additional subjects (e.g., Family Law, Secured Transactions).
Sources: [apps/web-svelte/src/lib/exam-types/mee/types.ts:6-20]()
## Summary
MEE Specific Components provide a highly structured framework for essay practice. By utilizing complex JSON-validated bundles and a multi-stage LLM runtime, the system can provide automated, IRAC-aligned feedback that mimics traditional bar exam grading while maintaining performance through deterministic evidence verification.
## Backend Systems
### V3.2 Engine & Compiler
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts)
- [apps/web-svelte/src/lib/server/v3_2/compiler/phases.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/compiler/phases.test.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts)
- [apps/web-svelte/scripts/benchmark-confidence.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-confidence.ts)
- [apps/web-svelte/scripts/benchmark-models.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-models.ts)
# V3.2 Engine & Compiler
The V3.2 Engine & Compiler represents the core automated grading and content transformation infrastructure for the MEE (Multistate Essay Examination) module. It provides a structured pipeline to transform "Canonical Packets"—which contain legal facts, rules, and rubric requirements—into executable scoring bundles used by the runtime engine to evaluate student submissions using Large Language Models (LLMs).
The system is designed to provide objective, multi-dimensional scoring through a deterministic aggregation of LLM-generated adherence checks, evidence verification, and rubric-based logic. It supports advanced features such as confidence-weighted scoring, complex issue dependency rules, and hard-band thresholds to ensure grading consistency and accuracy.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](), [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts]()
## Compiler Architecture & Phases
The compiler transforms a high-level `CanonicalPacket` and a `RubricSpec` into an `ExecutableScoringBundle`. This process occurs in distinct phases to ensure data normalization and the derivation of executable logic.
### Compilation Workflow
The compiler pipeline follows a sequence of normalization, derivation, and emission phases:
```mermaid
flowchart TD
P[Canonical Packet] --> N[Phase 1a: Normalize Packet]
N --> IB[Phase 1b: Derive Issue Bundles]
IB --> WE[Phase 2a: Emit Whole Essay Bundle]
WE --> AP[Phase 2b: Emit Aggregation Policy]
AP --> RP[Phase 2c: Emit Retrieval Policy]
RP --> HB[Phase 2d: Emit Hard Band Rules]
HB --> EC[Phase 2e: Emit Engine Compatibility]
EC --> EB[Executable Scoring Bundle]
```
The diagram shows the transformation of a raw packet into a scoring bundle through specific logical phases.
Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:6-265]()
### Key Compilation Phases
| Phase | Responsibility | Description |
| :--- | :--- | :--- |
| **Normalization** | `normalizePacket` | Deep-clones the packet and sorts recognition aliases and rule atoms for deterministic output. |
| **Issue Bundle Derivation** | `deriveIssueBundles` | Maps packet issues to specific dimensions (Spotting, Rule, Facts, Analysis, Conclusion) and assigns scoring scales. |
| **Whole Essay Emission** | `emitWholeEssayBundle` | Configures modifiers for responsiveness, coherence, off-point penalties, and incompleteness. |
| **Aggregation Policy** | `emitAggregationPolicy` | Validates band thresholds (0 to 1) and defines weights and calibration policies. |
| **Hard Band Rules** | `emitHardBandRules` | Generates deterministic triggers for maximum band caps (e.g., blank essays or missing required issues). |
Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:11-255](), [apps/web-svelte/src/lib/server/v3_2/compiler/phases.test.ts:18-185]()
## Runtime Grading Engine
The runtime engine executes the `gradeSubmission` function, which coordinates several asynchronous adapters to process a student's text against a compiled bundle.
### Grading Execution Flow
The grading process is an orchestrated pipeline of LLM calls and deterministic logic:
```mermaid
sequenceDiagram
participant R as Runtime Engine
participant P as Preprocessor
participant QC as Quality Channel (LLM)
participant E as Embedding Provider
participant AS as Adherence Scorer (LLM)
participant AG as Aggregator (Logic)
participant FR as Feedback Renderer (LLM)
R->>P: 1. Preprocess raw text
R->>QC: 2. Run Quality Channel
QC-->>R: Responsiveness/Coherence
R->>E: 3. Retrieve Spans (Embeddings)
E-->>R: Relevant text segments
loop For each Issue
R->>AS: 4. Check Packet Adherence
AS-->>R: Checklist results + Evidence
end
R->>R: 5. Verify Evidence (Deterministic)
R->>AG: 6. Aggregate Scores (Deterministic)
AG-->>R: Final Band + Component Scores
R->>FR: 7. Render Feedback
FR-->>R: Markdown Narrative
```
The diagram illustrates the sequential interaction between runtime components, highlighting where LLM calls occur versus deterministic logic.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:50-184]()
### Scoring Dimensions & Aggregation
The engine evaluates five distinct dimensions for every issue identified in the rubric:
* **Spotting**: Identifying the legal issue.
* **Rule**: Stating the applicable legal rule atoms.
* **Facts**: Applying relevant facts from the prompt.
* **Analysis**: Connecting rules to facts via reasoning.
* **Conclusion**: Reaching a legal conclusion.
The `aggregator` applies "caps" to issue scores. For example, a score may be capped if a required rule is missing or if the issue was not properly "spotted," regardless of the quality of the subsequent analysis.
Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:31-75](), [apps/web-svelte/scripts/benchmark-confidence.ts:213-248]()
## Data Models & Validation
The system relies on strict Zod schemas to ensure compatibility between packets, bundles, and the engine.
### Canonical Packet Schema
The `CanonicalPacket` is the source of truth for an exam question. It includes:
* **Prompt**: Facts and specific "calls of the question."
* **Issues**: Legal issues with `priority` (major/minor), `weight`, and `acceptable_paths` (required facts/rules).
* **Whole Essay Modifiers**: Gates for responsiveness and penalties for off-point content.
* **Band Exemplars**: Reference IDs for essays at specific performance bands (e.g., bands 2, 4, 6 for launch).
### Hard Band Rules
Hard band rules provide a safety net for automated grading, ensuring that certain conditions trigger an immediate maximum band limit.
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:121-236]()
| Rule ID | Condition | Trigger |
| :--- | :--- | :--- |
| `hbr_blank_or_irrelevant` | Blank/Irrelevant | Word count below `blank_threshold_word_count`. |
| `hbr_generic_minimal` | Generic/Minimal | Composite of low word count and low verified issue hits. |
| `hbr_top_band_[ID]` | Missing Top Requirement | Missing an issue marked as `required_for_top_band`. |
Sources: [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:221-255]()
## Benchmarking & Calibration
The system includes scripts for benchmarking model performance (`benchmark-models.ts`) and calibrating scoring methods (`benchmark-confidence.ts`).
### Confidence Calibration
The V3.2 engine supports confidence-weighted scoring. When an LLM evaluates a checklist item (e.g., "Did the student state the rule?"), it provides a confidence score. The aggregator uses this to adjust the credit given for that item. Benchmarking compares:
1. **Binary Scoring**: Items are either 0% or 100% credit based on pass/fail.
2. **Confidence-Weighted**: Credit is adjusted by the LLM's self-reported confidence.
```typescript
// Example of binary vs confidence aggregation logic
const binaryAdherence = adherenceResults.map((a) => ({
...a,
checklist_results: a.checklist_results.map((cr) => ({
...cr,
confidence: cr.pass ? 1.0 : cr.confidence
}))
}));
```
Sources: [apps/web-svelte/scripts/benchmark-confidence.ts:114-131](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:153-157]()
## Conclusion
The V3.2 Engine & Compiler provides a robust framework for automated legal essay grading. By separating the content definition (`CanonicalPacket`) from the grading logic (`ExecutableScoringBundle`), the system allows for flexible rubric adjustments and model benchmarking without changing the underlying exam content. The combination of multi-LLM-channel analysis and deterministic evidence verification ensures that student feedback is both detailed and grounded in the specific facts and rules of the packet.
### Grading Run Workflow
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/src/lib/server/v3_2/grading-run-writer.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/grading-run-writer.ts)
- [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/packet/%5BpacketId%5D/+page.server.ts)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts)
# Grading Run Workflow
The **Grading Run Workflow** represents the core analytical pipeline of the Bar Prep by SHEP platform, specifically within the V3.2 grading engine. It manages the transformation of raw student submissions into structured, rubric-aligned feedback through a series of deterministic and LLM-augmented stages. The workflow ensures that legal essays are evaluated for quality, issue recognition, and evidence adherence while maintaining a strict runtime budget.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:35-40](), [apps/web-svelte/src/lib/server/v3_2/grading-run-writer.ts:1-10]()
## Core Architecture and Execution
The grading process is encapsulated in the `gradeSubmission` function, which orchestrates several specialized modules including preprocessing, retrieval, adherence checking, and aggregation. The system uses a "Runtime Adapter" pattern to abstract external dependencies like LLM calls (embedding, quality checks, feedback generation) and database persistence.
### Execution Lifecycle
The workflow follows a strictly ordered sequence of operations to ensure data integrity and budget control:
```mermaid
flowchart TD
Start([Start Grading]) --> Validate[Compatibility Check]
Validate --> Preprocess[Text Preprocessing]
Preprocess --> Quality[Quality Channel - LLM]
Quality --> Embed[Embedding/Retrieval]
Embed --> Adherence[Packet Adherence - LLM per Issue]
Adherence --> Evidence[Evidence Verification - Deterministic]
Evidence --> Aggregate[Scoring Aggregation - Deterministic]
Aggregate --> Feedback[Feedback Rendering - LLM]
Feedback --> Persist[Database Persistence]
Persist --> End([Result Returned])
```
The workflow includes explicit checks for engine version and model manifest compatibility before execution begins.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:50-70](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:75-160]()
## Pipeline Stages
### 1. Preprocessing and Quality Analysis
The submission text is first normalized and segmented into sentences and paragraphs based on a `segmentation_policy`. Immediately following, the **Quality Channel** performs a single LLM call to assess high-level metrics such as responsiveness, coherence, and incompleteness.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:77-88](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts:130-136]()
### 2. Retrieval and Adherence
The system identifies candidate text spans relevant to specific legal issues using embeddings. For each issue identified in the `ExecutableScoringBundle`, a **Packet Adherence** check is performed. This stage involves one LLM call per issue to determine if the student recognized the issue and to extract quoted evidence.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:98-115](), [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts:143-155]()
### 3. Evidence Verification and Aggregation
Evidence verification is a deterministic process that matches LLM-extracted quotes against the preprocessed text using normalized substring matching. The **Aggregator** then calculates scores based on:
* Issue recognition status.
* Checklist item pass/fail results.
* Verification of required evidence.
* Application of hard-band rules and issue caps (e.g., capping a score if a required rule is missing).
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:117-156](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts:75-95]()
### 4. Feedback Rendering
The final analytical stage uses the aggregated scores and adherence rationales to generate a structured feedback report. This typically involves a narrative markdown response that explains the student's performance across different dimensions.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:162-172](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts:145-150]()
## Data Persistence Model
The workflow concludes by persisting the results to the database via `writeGradingRun`. This operation is transactional, ensuring that the grading run header, individual issue findings, and evidence records are saved atomically.
### Database Schema Relationships
```mermaid
erDiagram
GRADING_RUN ||--o{ ISSUE_FINDING : contains
GRADING_RUN ||--o{ EVIDENCE_RECORD : references
GRADING_RUN {
uuid id
uuid submission_id
jsonb grading_run_json
string engine_version
decimal cost_usd
}
ISSUE_FINDING {
string issue_id
boolean recognized
decimal raw_issue_score
string strength
}
EVIDENCE_RECORD {
string evidence_id
string quote_text
boolean verified
string verification_result
}
```
Tables utilize a dual-storage strategy: denormalized columns for query performance and a full `jsonb` column for lossless data replay.
Sources: [apps/web-svelte/src/lib/server/v3_2/grading-run-writer.ts:1-25](), [apps/web-svelte/database/migrations/001_initial_schema.sql:165-200]()
## Component Summary Table
| Component | Type | Description | LLM Requirement |
| :--- | :--- | :--- | :--- |
| **Preprocessing** | Deterministic | Normalizes text and segments into sentences/paragraphs. | No |
| **Quality Channel** | Analytic | Evaluates overall essay responsiveness and coherence. | Yes (1 call) |
| **Retrieval** | Analytic | Finds relevant text spans using embeddings. | Yes (1 call) |
| **Packet Adherence** | Analytic | Evaluates specific legal issues and extracts evidence. | Yes (1 per issue) |
| **Aggregator** | Deterministic | Computes final band and component scores. | No |
| **Feedback Renderer** | Analytic | Generates human-readable critique in Markdown. | Yes (1 call) |
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:75-175](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts:153-162]()
## Error Handling and Validation
The workflow implements strict validation for `ExecutableScoringBundle` requirements. It throws errors if:
* The `ENGINE_VERSION` is outside the allowed range (`min_engine_version` to `max_engine_version`).
* The provided `ModelManifest` is not explicitly allowlisted in the bundle's `compatible_model_manifests`.
* A database transaction fails during the write phase, which triggers a full rollback of the grading run data.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:50-65](), [apps/web-svelte/src/lib/server/v3_2/grading-run-writer.ts:1-5]()
The Grading Run Workflow ensures that every bar exam essay is evaluated through a reproducible, transparent, and architecturally sound pipeline, balancing advanced LLM analysis with deterministic scoring logic.
### Certification & Quality Gates
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3\_2/certification/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/certification/index.ts)
- [apps/web-svelte/src/lib/mee/v3\_2/schemas/benchmark-suite.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts)
- [apps/web-svelte/src/lib/mee/v3\_2/regression/__integration__/quality-gate.integration.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/regression/__integration__/quality-gate.integration.test.ts)
- [apps/web-svelte/database/migrations/006\_v3\_2\_canonical\_tables.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql)
- [apps/web-svelte/src/lib/server/v3\_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
# Certification & Quality Gates
The Certification and Quality Gates system is a multi-layered validation framework designed to ensure the technical accuracy, stability, and grading reliability of the MEE (Multistate Essay Examination) grading engine. It operates within the V3.2 runtime context, providing a mechanism to "certify" compiled scoring bundles against human-vetted benchmarks before they are published for student use. Sources: [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql:11-17](), [apps/web-svelte/src/lib/server/v3_2/certification/index.ts:1-10]()
The scope of this system encompasses absolute case pass rates, evidence verification accuracy, and cross-subject sanity checks. It ensures that any changes to the grading logic, model manifests, or prompt versions do not result in regression of grading quality. Sources: [apps/web-svelte/src/lib/mee/v3_2/regression/__integration__/quality-gate.integration.test.ts:41-55](), [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:133-143]()
## Certification Architecture
The certification process involves executing the grading engine on a set of "anchors" (exemplar responses with known bands) and comparing the engine's output against the expected results defined in a `BenchmarkSuite`.
```mermaid
flowchart TD
A[Benchmark Suite] --> B[Certification Runner]
C[Executable Bundle] --> B
D[Model Manifest] --> B
B --> E{Threshold Evaluation}
E -- Pass --> F[Certified Status]
E -- Fail --> G[Failure Diff / Resolutions]
```
The diagram shows the inputs required for the certification runner and the resulting flow based on threshold evaluation. Sources: [apps/web-svelte/src/lib/server/v3_2/certification/index.ts:25-30](), [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql:240-255]()
### Core Components
* **Benchmark Suite:** A collection of anchors, adversarial cases, and pairwise cases used for validation. Anchors typically cover specific score bands (e.g., 2, 4, 6 for the `llm_only_launch` profile). Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:145-165]()
* **Certification Report:** The artifact generated after running certification, containing pass/fail status, metrics like `max_band_error`, and `failure_diffs`. Sources: [apps/web-svelte/src/lib/server/v3_2/certification/index.ts:11-23]()
* **Thresholds:** Deterministic limits that the report metrics must satisfy for the bundle to be considered "certified." Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:116-126]()
## Quality Gate Integration
The Quality Gate represents the final automated hurdle in the deployment pipeline. It integrates multiple sub-gates to provide a "Full Pass" status for a set of packets. Sources: [apps/web-svelte/src/lib/mee/v3_2/regression/__integration__/quality-gate.integration.test.ts:37-40]()
| Sub-Gate | Purpose | Metric/Criterion |
| :--- | :--- | :--- |
| **Per-Packet Gate** | Validates individual packet stability. | Certification status + Evidence verification rate. |
| **Self-Consistency** | Ensures the model provides stable results over multiple runs. | Decision (Pass/Fail) based on variance. |
| **Inter-Model Agreement** | Checks consistency across different LLM manifests. | Agreement status. |
| **Budget Gate** | Ensures LLM calls and costs remain within acceptable bounds. | Mean LLM calls + Mean spend cents. |
| **Cross-Subject Sanity** | Detects drift when a model graded on one subject is tested on another. | Band delta between own vs other subject. |
Sources: [apps/web-svelte/src/lib/mee/v3_2/regression/__integration__/quality-gate.integration.test.ts:43-65](), [apps/web-svelte/database/schema.sql:560-575]()
## Data Models and Schemas
The system relies on specific database tables to track the lifecycle of certified artifacts and their validation history.
### Certification Persistence
```mermaid
erDiagram
CERTIFICATION_REPORTS ||--o{ CERTIFICATION_RESOLUTIONS : "contains"
CERTIFICATION_REPORTS {
uuid id
text bundle_id
text engine_version
boolean pass
jsonb summary_metrics
}
CERTIFICATION_RESOLUTIONS {
uuid id
text case_id
text resolution_kind
text notes
}
```
The relationship between reports and resolutions allows operators to manually resolve specific failures in the benchmark set. Sources: [apps/web-svelte/database/migrations/006_v3_2_canonical_tables.sql:240-265](), [apps/web-svelte/database/schema.sql:215-245]()
### Benchmark Suite Configuration
The `BenchmarkSuiteSchema` defines the requirements for validation data. For the `llm_only_launch` profile, it strictly requires exactly 3 anchors at bands 2, 4, and 6.
```typescript
// apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:169-181
if (b.schema_profile === 'llm_only_launch') {
const bands = b.anchors.map((a) => a.band).sort((x, y) => x - y);
if (b.anchors.length !== 3 || bands.join(',') !== '2,4,6') {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ['anchors'],
message: 'llm_only_launch requires exactly 3 anchors at bands 2, 4, 6'
});
}
}
```
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:169-181]()
## Validation Logic
The `certifyBundle` function executes the `gradeSubmission` runtime for every anchor in the suite. It disables database persistence during this phase to ensure that certification runs do not pollute student submission records. Sources: [apps/web-svelte/src/lib/server/v3_2/certification/index.ts:32-45]()
### Threshold Evaluation Logic
The engine evaluates the following criteria to determine a final "pass" status:
1. **Anchor Pass Rate:** `passRate >= thresholds.min_absolute_case_pass_rate`
2. **Evidence Quality:** `quoteFailureRate <= thresholds.max_quote_verification_failure_rate`
3. **Band Accuracy:** `maxBandError <= thresholds.max_band_error`
4. **Confidence:** `avgConfidence >= thresholds.min_average_confidence`
Sources: [apps/web-svelte/src/lib/server/v3_2/certification/index.ts:114-118]()
## Conclusion
Certification and Quality Gates serve as the primary defensive layer for the Bar Prep system's automated grading. By enforcing rigorous thresholds on anchor pass rates and evidence verification, the system ensures that the legal analysis provided to students is accurate and consistent with human-authored standards. The integration of budget and drift checks further protects the system from economic inefficiency and cross-domain logic failures. Sources: [apps/web-svelte/src/lib/mee/v3_2/regression/__integration__/quality-gate.integration.test.ts:80-110](), [apps/web-svelte/src/lib/server/v3_2/certification/index.ts:120-140]()
## Model Integration
### Rubric Scoring Engine
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/aggregator.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/aggregator.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/rubric-spec.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/rubric-spec.ts)
- [apps/web-svelte/content/mee/rubric-default.json](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/content/mee/rubric-default.json)
- [apps/web-svelte/src/lib/server/v3_2/runtime/channels/quality.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/channels/quality.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/channels/packet-adherence.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/channels/packet-adherence.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts)
# Rubric Scoring Engine
The Rubric Scoring Engine is a sophisticated evaluation system designed to grade legal essay submissions, specifically for the Multistate Essay Examination (MEE) and NextGen Bar Prep. It utilizes a multi-channel LLM architecture to analyze candidate responses against structured `RubricSpec` definitions. The engine decomposes an essay into specific issues, evaluates them across multiple dimensions (Spotting, Rule, Facts, Analysis, Conclusion), and applies deterministic aggregation logic to produce a final numerical score and performance band.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:1-50](), [apps/web-svelte/src/lib/mee/v3_2/schemas/rubric-spec.ts:1-20]()
## Architecture and Data Flow
The scoring engine operates as a sequential pipeline that combines asynchronous LLM calls with deterministic verification and aggregation. The process begins with text preprocessing and ends with the generation of a `GradingRun` record.
### Execution Pipeline
1. **Preprocessing:** Normalizes input text and segments it into atomic units (sentences/paragraphs) for evidence tracking.
2. **Quality Channel:** Analyzes global essay traits such as responsiveness, coherence, and incompleteness.
3. **Retrieval:** Uses embeddings to identify relevant spans for each issue defined in the rubric.
4. **Packet Adherence:** A per-issue LLM analysis that checks if specific checklist items (rubric criteria) were met.
5. **Evidence Verification:** Deterministically verifies that LLM-quoted evidence actually exists in the student's submission.
6. **Aggregation:** Computes the final score by applying weights, caps, and whole-essay modifiers.
```mermaid
flowchart TD
Input[Student Submission] --> Prep[Preprocessing]
Prep --> Quality[Quality Channel - Global Metrics]
Prep --> Embed[Embedding & Retrieval]
Embed --> Adherence[Packet Adherence - Per Issue]
Adherence --> Evidence[Evidence Verification]
Quality --> Agg[Aggregation Engine]
Evidence --> Agg
Agg --> Result[Final Score & Band]
Result --> Feedback[Feedback Generation]
```
The diagram shows the sequential flow from raw text input to the final score calculation.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:50-150]()
## Scoring Components and Dimensions
The engine evaluates issues based on five core dimensions. Each dimension is assigned a weight within the `RubricSpec` to determine its contribution to the overall issue score.
### Evaluation Dimensions
| Dimension | Description | Typical Weight |
| :--- | :--- | :--- |
| **Spotting** | Identification of the relevant legal issue. | 0.15 - 0.20 |
| **Rule** | Statement of the applicable legal rule or doctrine. | 0.20 - 0.25 |
| **Facts** | Identification and inclusion of relevant facts from the prompt. | 0.20 |
| **Analysis** | Application of the rule to the facts. | 0.30 - 0.35 |
| **Conclusion** | Reaching a legally sound conclusion. | 0.10 |
Sources: [apps/web-svelte/content/mee/rubric-default.json:6-12](), [apps/web-svelte/src/lib/server/v3_2/runtime/aggregator.ts:1-20]()
## Aggregation Logic
The `aggregator.ts` module is responsible for the deterministic calculation of the final score. It applies several layers of logic to ensure the grade is robust and penalized appropriately for major omissions.
### Issue Scoring and Caps
Each issue is scored by summing the weighted performance across the five dimensions. However, "Caps" are applied to limit the maximum possible score of an issue if critical elements are missing:
* **max_if_not_spotted:** Prevents high scores if the primary issue wasn't identified.
* **max_if_missing_required_rule:** Limits the score if a fundamental rule statement is absent.
* **max_if_missing_required_analysis:** Penalizes heavily if the student fails to apply the law to facts.
### Whole Essay Modifiers
Global quality metrics can adjust the final score or enforce "Gates":
* **Responsiveness Gate:** If an essay is "blank" or "irrelevant," it is capped at the lowest performance band (Band 1).
* **Coherence Modifier:** Adds or subtracts small percentages (e.g., +/- 3%) based on structural quality.
* **Incompleteness Cap:** Limits the maximum score if the essay is significantly unfinished.
```mermaid
graph TD
SubScores[Dimension Scores] --> WeightedSum[Weighted Issue Sum]
WeightedSum --> Caps[Apply Issue Caps]
Caps --> IssueTotal[Total Issue Score]
IssueTotal --> GlobalMod[Apply Global Modifiers]
GlobalMod --> Banding[Map to Band 1-6]
```
The diagram illustrates how individual dimension scores are refined through caps and global modifiers before being mapped to a final band.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/aggregator.ts:50-120](), [apps/web-svelte/content/mee/rubric-default.json:13-20]()
## Configuration: RubricSpec
The `RubricSpecSchema` defines the structure of a rubric version. It contains the weights, thresholds, and policies used during a grading run.
### Key Configuration Fields
| Field | Type | Description |
| :--- | :--- | :--- |
| `component_weights` | Object | Weights for spotting, rule, facts, analysis, and conclusion. |
| `band_thresholds` | Array | Defines the score ranges for Performance Bands 1 through 6. |
| `modifier_values` | Object | Numerical penalties or bonuses for global metrics like coherence. |
| `length_policy` | Object | Minimum word counts and penalties for excessive brevity. |
| `calibration_policy` | Object | Determines how thresholds are anchored (e.g., `exemplar_anchored_thresholds`). |
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/rubric-spec.ts:10-80](), [apps/web-svelte/content/mee/rubric-default.json:20-60]()
## Evidence Verification
To prevent LLM "hallucinations" (where the model claims the student said something they didn't), the engine includes a `verifyEvidence` step. The LLM must provide quoted strings from the student's text. The engine then performs a deterministic search (often normalized substring matching) to confirm the quote's existence.
* **Min Verified Quotes:** A rubric item may require a minimum number of verified quotes to receive full credit.
* **Max Score Without Evidence:** Limits the score of a rubric item if no valid evidence is found, even if the LLM marks it as "passing."
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:1-40](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:110-135]()
## Performance Banding
The engine maps the final calibrated score to one of six performance bands. These bands provide a standardized assessment of the student's proficiency.
| Band | Label | Typical Score Range |
| :--- | :--- | :--- |
| **6** | Excellent | 0.80 - 1.00 |
| **5** | Strong | 0.65 - 0.80 |
| **4** | Competent | 0.50 - 0.65 |
| **3** | Developing | 0.35 - 0.50 |
| **2** | Minimal | 0.20 - 0.35 |
| **1** | Non-Responsive| 0.00 - 0.20 |
Sources: [apps/web-svelte/content/mee/rubric-default.json:28-40]()
The Rubric Scoring Engine provides a rigorous, automated framework for grading complex legal analysis, ensuring consistency through structured schemas and deterministic verification while leveraging LLMs for nuanced semantic evaluation.
### AI Prompt Engineering & Agents
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3_2/authoring/prompts/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/authoring/prompts/index.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/channels/packet-adherence.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/channels/packet-adherence.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/channels/quality.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/channels/quality.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/feedback-renderer.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/feedback-renderer.ts)
- [apps/web-svelte/src/lib/server/v3_2/authoring/prompts/index.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/authoring/prompts/index.test.ts)
- [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/packet/%5BpacketId%5D/+page.server.ts)
# AI Prompt Engineering & Agents
The AI Prompt Engineering and Agents system within Bar Prep by SHEP is a multi-stage pipeline designed to evaluate legal essays, provide structured scoring, and generate qualitative feedback. The architecture relies on specialized "channels" or agents—specifically for Quality, Packet Adherence (Scoring), and Feedback—that interact with Large Language Models (LLMs) via an AI Gateway.
This system is central to the project's ability to provide automated, rubric-aligned grading for MEE (Multistate Essay Examination) and NextGen UBE tasks, using stable, versioned prompt templates and model manifests to ensure consistent output.
## Prompt Management and Authoring
Prompts are treated as versioned assets. The system uses a centralized loader to manage markdown-based instruction files, ensuring that each prompt used by an agent has a stable identity and verifiable integrity through SHA-256 hashing.
### Canonical Prompt Templates
The system identifies three primary authoring-time prompt types:
* **Drafting Agent:** Used for initial content generation.
* **Anchor Regeneration:** Used for updating exemplar answers.
* **Packet Revision:** Used for iterative improvements to assessment packets.
Sources: [apps/web-svelte/src/lib/server/v3_2/authoring/prompts/index.ts:7-11](), [apps/web-svelte/src/lib/server/v3_2/authoring/prompts/index.test.ts:10-15]()
### Implementation Detail: Prompt Loader
The `loadPromptTemplate` function resolves file paths, reads markdown content, and generates a hash for stability tracking.
```typescript
export async function loadPromptTemplate(name: PromptName): Promise {
// ... validation logic ...
const text = await readFile(resolve(here, `${name}.md`), 'utf-8');
const hash = createHash('sha256').update(text, 'utf-8').digest('hex');
return { name, text, hash };
}
```
Sources: [apps/web-svelte/src/lib/server/v3_2/authoring/prompts/index.ts:22-29]()
## Grading Pipeline Architecture
The grading process is an orchestrated flow involving multiple LLM calls across different specialized agents. This is coordinated by the `gradeSubmission` runtime, which utilizes adapters to communicate with the AI Gateway.
### Agent Workflow
The following sequence diagram illustrates the data flow from submission to the final structured result:
```mermaid
sequenceDiagram
participant R as Runtime (gradeSubmission)
participant Q as Quality Agent
participant A as Adherence Agent
participant F as Feedback Agent
participant G as AI Gateway (LLM)
R->>Q: runQualityChannel(submission)
Q->>G: generateObject(QualitySchema)
G-->>Q: Quality Findings
Q-->>R: Findings List
loop Per Legal Issue
R->>A: runPacketAdherence(IssueBundle)
A->>G: generateObject(AdherenceSchema)
G-->>A: Checklist Results
A-->>R: Issue Result
end
R->>F: feedbackRenderer(All Results)
F->>G: generateObject(FeedbackSchema)
G-->>F: Markdown Narratives
F-->>R: Final Feedback Body
```
The diagram shows how the Runtime coordinates sequential calls to specialized agents, each returning structured data validated by Zod schemas.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts:117-155](), [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts:145-185]()
## Specialized Agents (Channels)
The system breaks down evaluation into distinct "channels," each governed by specific Zod schemas and prompt-building logic.
### 1. Quality Agent
Evaluates high-level metrics of the essay such as responsiveness, coherence, and off-point content. It uses a "responsiveness gate" to determine if a submission is even relevant to the prompt before deeper analysis occurs.
| Metric | Levels | Description |
| :--- | :--- | :--- |
| **Responsiveness** | blank, irrelevant, generic_minimal, responsive, adequate | Determines if the student answered the call of the question. |
| **Coherence** | strong, neutral, weak | Evaluates logical organization and transitions. |
| **Off-Point** | none, moderate, severe | Penalizes irrelevant legal analysis or material. |
| **Incompleteness** | none, moderate, severe | Identifies truncated or unfinished responses. |
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/channels/quality.ts:6-12](), [apps/web-svelte/src/lib/server/v3_2/runtime/channels/quality.ts:44-50]()
### 2. Packet Adherence (Scoring) Agent
This agent is responsible for the granular legal scoring. It maps the student's text against a specific "Issue Bundle" from the assessment packet. For every legal issue (e.g., "Mutual Mistake" or "Statute of Frauds"), the agent performs a checklist-based evaluation.
**Key Logic:**
* **Prompt Constraints:** The system prompt explicitly instructs the agent to only quote text directly addressing the specific issue being evaluated to prevent "cross-attribution" errors.
* **Structured Output:** Returns `checklist_results` including a pass/fail boolean, a confidence score, and the specific `quoted_evidence` from the student's text.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/channels/packet-adherence.ts:18-24](), [apps/web-svelte/src/lib/server/v3_2/runtime/channels/packet-adherence.ts:68-75]()
### 3. Feedback Agent
The Feedback Agent synthesizes the findings from the Quality and Adherence agents into human-readable markdown narratives. It generates:
* **Issue Narratives:** Specific feedback for each legal issue identified (or missed).
* **Overall Narrative:** A high-level summary of the student's performance.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/feedback-renderer.ts:10-15]()
## AI Gateway and Model Manifests
The project uses a Gateway pattern (`@ai-sdk/gateway`) to decouple the grading logic from specific LLM providers (e.g., OpenAI, Anthropic).
### Model Manifest Configuration
The `ModelManifest` defines which specific models are used for each task in the pipeline.
| Task | Provider Example | Model Example |
| :--- | :--- | :--- |
| **Checklist Scorer** | openai | gpt-4o-mini |
| **Feedback Renderer** | openai | gpt-4o-mini |
| **Embedding** | openai | text-embedding-3-small |
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts:100-106](), [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts:125-135]()
### Resolution Logic
Before execution, the `resolveModel` function ensures the requested model manifest is compatible with the `ExecutableScoringBundle` version and verifies data retention postures.
Sources: [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts:127-133]()
## Conclusion
The AI Prompt Engineering and Agents system provides a structured, multi-agent approach to legal essay evaluation. By separating quality checks, rubric adherence, and feedback generation into specialized channels, the system ensures technical accuracy while maintaining the flexibility to swap underlying LLM models via the AI Gateway. This architecture supports the project's goal of mirroring real bar exam grading conditions through automated, high-confidence scoring.
### Self-Consistency Validation Loop
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.test.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
- [apps/web-svelte/scripts/benchmark-confidence.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-confidence.ts)
- [apps/web-svelte/scripts/benchmark-models.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-models.ts)
# Self-Consistency Validation Loop
The Self-Consistency Validation Loop is a critical quality assurance mechanism within the MEE (Multistate Essay Examination) grading runtime. Its primary purpose is to ensure that the LLM-based grading pipeline produces stable, reproducible, and accurate results across multiple iterations or different model configurations. By comparing multiple grading runs against established benchmarks and model manifests, the system validates that a specific "Packet" and "Scoring Bundle" are fit for production use.
This loop is integrated into the broader certification pipeline, where a `self_consistency_report` acts as a prerequisite for final packet publication. It specifically addresses variance in LLM outputs, such as confidence scores and issue recognition, to maintain the integrity of the 6-band grading scale.
Sources: [apps/web-svelte/database/schema.sql:475-484](), [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:121-123]()
## Architectural Overview
The validation loop operates by executing the `gradeSubmission` function across a series of adversarial and absolute test cases defined in a `BenchmarkSuite`. The results are then aggregated into a `self_consistency_reports` entry, which determines if the scoring logic meets the required stability thresholds.
### Core Components
| Component | Description |
| :--- | :--- |
| **Benchmark Suite** | Defines anchors (bands 1-6), absolute cases, and adversarial archetypes used for testing. |
| **Model Manifest** | Specifies the exact LLM versions (embedding, scorer, renderer) used during the loop. |
| **Grading Runtime** | The `gradeSubmission` engine that executes preprocessing, retrieval, and adherence channels. |
| **Stability Policy** | Defines the `runs_per_case` and the maximum allowed score/band variance. |
| **Quality Gate** | Final automated check that evaluates certification, self-consistency, and agreement rates. |
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:145-165](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:41-47](), [apps/web-svelte/database/schema.sql:509-518]()
### Logic Flow
The following diagram illustrates the sequence of operations within the validation loop, from initial preprocessing to the final self-consistency decision.
```mermaid
flowchart TD
Start[Start Validation Loop] --> Prep[Preprocess Essay Text]
Prep --> Quality[Run Quality Channel]
Quality --> Embed[Generate Embeddings]
Embed --> Retrieve[Retrieve Candidate Spans]
Retrieve --> Adherence[Run Packet Adherence Loop]
Adherence --> Verify[Deterministic Evidence Verification]
Verify --> Agg[Aggregate Component Scores]
Agg --> Report[Generate Self-Consistency Report]
Report --> Decision{Decision: Pass/Fail?}
Decision -->|Pass| Cert[Certified for Publication]
Decision -->|Fail| Review[Manual Review/Resolution]
```
The diagram shows the sequential execution of the v3.2 runtime pipeline, culminating in a pass/fail consistency decision based on aggregated results.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:69-140](), [apps/web-svelte/database/schema.sql:725-737]()
## Data Models and Schemas
### Self-Consistency Report Structure
Reports are stored in the `self_consistency_reports` table and validated via Zod schemas. They track the consumption of "attempts" against a configured limit.
* **Report ID**: Unique identifier for the validation session.
* **Packet/Bundle Refs**: Links the report to specific versions of content and scoring logic.
* **Runs (JSONB)**: Contains the raw results of multiple iterations to detect drift.
* **Decision**: An enum of `pass` or `fail`.
* **Attempts**: Tracking `attempts_consumed` vs `attempts_allowed`.
Sources: [apps/web-svelte/database/schema.sql:725-737](), [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:121-123]()
### Stability and Certification Thresholds
The `BenchmarkSuite` schema defines the specific constraints that the validation loop must satisfy.
```typescript
const StabilityPolicySchema = z.object({
runs_per_case: z.number().int().min(1),
max_score_variance: z.number().min(0).max(1),
max_band_variance: z.number().int().min(0).max(6)
});
const CertificationThresholdsSchema = z.object({
min_absolute_case_pass_rate: z.number().min(0).max(1),
max_quote_verification_failure_rate: z.number().min(0).max(1),
min_average_confidence: z.number().min(0).max(1)
});
```
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:103-119]()
## Execution Stages
### 1. Adversarial Testing
The loop utilizes adversarial archetypes to test the robustness of the `checklist_scorer`. These include:
* `verbose_rule_dump`: Testing if the model incorrectly rewards length over analysis.
* `legalese_off_point`: Ensuring the `QualityChannel` correctly flags irrelevant content.
* `contradictory_issue_treatment`: Checking if the model can handle conflicting statements.
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:81-93]()
### 2. Multi-Model Agreement
As evidenced in the `benchmark-models.ts` script, the loop can be extended to compare results across different model families (e.g., GPT-4o-mini vs Gemini 3.1 Flash). This ensures that the scoring logic is not over-fitted to a specific model's behavior. It measures:
* **Band/Score Delta**: Variance between different LLMs on the same essay.
* **Attribution Agreement**: Whether different models identify the same candidate spans for a legal issue.
Sources: [apps/web-svelte/scripts/benchmark-models.ts:101-115](), [apps/web-svelte/scripts/benchmark-models.ts:311-325]()
### 3. Quality Gate Verification
The final stage of the loop involves a deterministic "Quality Gate" that aggregates multiple metrics into a boolean `pass` status.
| Metric | Threshold/Condition |
| :--- | :--- |
| **Certification** | Must be 'pass' |
| **Self-Consistency** | Must be 'pass' |
| **Inter-Model Agreement** | Must be 'pass' |
| **Evidence Verification Rate** | Must be >= 0.9 (90%) |
Sources: [apps/web-svelte/database/schema.sql:509-518]()
## Benchmarking and Calibration
The validation loop uses `benchmark-confidence.ts` to calibrate the difference between binary scoring and confidence-weighted aggregation. This ensures that the `aggregate` function correctly applies caps (e.g., `max_if_not_spotted`) consistently across runs.
```mermaid
sequenceDiagram
participant Script as "Benchmark Script"
participant Runtime as "Grading Runtime"
participant Agg as "Aggregator"
Script->>Runtime: gradeSubmission(input, bundle)
Runtime->>Agg: aggregate(adherenceResults, policy)
Agg-->>Runtime: confidenceWeightedResult
Script->>Agg: binaryAggregate(overriddenResults)
Agg-->>Script: binaryResult
Note over Script,Agg: Compare Delta (Band & Score)
```
The sequence diagram shows how the benchmarking scripts isolate scoring differences by re-running aggregation with modified confidence values.
Sources: [apps/web-svelte/scripts/benchmark-confidence.ts:98-154](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:133-143]()
## Conclusion
The Self-Consistency Validation Loop provides the necessary empirical data to certify MEE packets. By automating the execution of benchmark suites across multiple iterations and models, it minimizes the risk of non-deterministic LLM behavior affecting student scores, ensuring that every published packet meets the project's v3.2 quality standards.
### Model Manifest & Transport
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3_2/model-manifest-writer.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/model-manifest-writer.test.ts)
- [apps/web-svelte/src/lib/server/v3_2/llm-transport.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/llm-transport.test.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/routes/mee/practice/packet/%5BpacketId%5D/+page.server.ts)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
- [apps/web-svelte/scripts/benchmark-models.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-models.ts)
# Model Manifest & Transport
The **Model Manifest & Transport** system serves as the abstraction layer between the Bar Prep application's grading logic and external Large Language Model (LLM) providers. It defines a deterministic configuration for AI models (manifests) and a secure routing mechanism (transport) to ensure that specific tasks—such as embedding, checklist scoring, and feedback rendering—are performed by authorized and validated model versions.
This system is critical for maintaining "zero retention" data postures and ensuring that the grading pipeline remains consistent across different exam packets and runtime environments.
## Architecture and Data Flow
The architecture relies on a "Manifest" to define the specific AI infrastructure for a grading run. The transport layer then resolves these definitions into actionable gateway strings while enforcing security policies such as allowlisting and data retention checks.
### Grading Runtime Integration
The transport layer is invoked during the `gradeSubmission` process. It resolves specific roles to model IDs which are then used by adapters to communicate with the AI gateway.
```mermaid
flowchart TD
A[Grading Runtime] --> B{Resolve Model}
B --> C[Check Allowlist]
B --> D[Check Retention Posture]
C -->|Pass| E[Get Routing String]
D -->|Pass| E
E --> F[AI Gateway]
F --> G[Provider: OpenAI/Google/etc.]
subgraph Manifest Definitions
H[Checklist Scorer]
I[Feedback Renderer]
J[Embedding Model]
end
H & I & J -.-> B
```
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:70-130](), [apps/web-svelte/src/lib/server/v3_2/llm-transport.test.ts:37-56]()
## Model Manifest Structure
A **Model Manifest** is a versioned configuration object that pins specific AI models to roles within the system. This ensures that a grading result can be audited against the exact model versions used to produce it.
### Core Components
| Field | Type | Description |
|-------|------|-------------|
| `model_manifest_id` | `string` | Unique identifier (e.g., `mm_default_launch`). |
| `embedding` | `ModelPin` | Config for text embedding models. |
| `checklist_scorer` | `ModelPin` | The primary model for grading checklist adherence. |
| `feedback_renderer` | `ModelPin` | The model responsible for generating student-facing feedback. |
| `data_retention_mode` | `v3_2_retention_mode` | Options: `zero_retention`, `provider_default`, `internal_only`. |
Sources: [apps/web-svelte/src/lib/server/v3_2/model-manifest-writer.test.ts:10-18](), [apps/web-svelte/database/schema.sql:565-580]()
### Database Schema
The `model_manifests` table stores these configurations, linking them to retention posture checks to guarantee compliance before execution.
```mermaid
erDiagram
model_manifests {
uuid id
text model_manifest_id
jsonb embedding
jsonb checklist_scorer
jsonb feedback_renderer
jsonb prompt_versions
v3_2_retention_mode data_retention_mode
uuid retention_posture_check_id
text status
}
retention_posture_checks {
uuid id
text model_manifest_id
jsonb model_pins_snapshot
v3_2_retention_verdict verdict
timestamptz checked_at
}
model_manifests ||--o| retention_posture_checks : "validated by"
```
Sources: [apps/web-svelte/database/schema.sql:565-595]()
## Transport and Resolution Logic
The Transport layer is responsible for converting a high-level manifest role into a provider-specific routing string.
### Resolve Model Logic
The function `resolveModel` performs the following validations:
1. **API Key Presence**: Verifies `AI_GATEWAY_API_KEY` exists in the environment.
2. **Allowlist Check**: Ensures the manifest is explicitly allowed for the specific bundle being graded.
3. **Retention Posture**: Confirms that the model's data retention configuration meets the required security standards (e.g., `zero_retention`).
### Transport Error Codes
When resolution fails, the system throws a `TransportError` with specific codes:
* `missing_api_key`: The gateway key is not configured.
* `manifest_not_allowlisted`: The bundle does not support the requested manifest.
* `retention_posture_missing`: No valid retention check exists for the model configuration.
Sources: [apps/web-svelte/src/lib/server/v3_2/llm-transport.test.ts:70-105]()
## Implementation Example: Grading Submission
In the server-side submission handler, models are resolved before the grading pipeline is initiated.
```typescript
// Resolve models for the grading run
const [checklistRoutingId, feedbackRoutingId, embeddingRoutingId] = await Promise.all([
resolveModel(manifest, 'checklist_scorer', resolveCtx),
resolveModel(manifest, 'feedback_renderer', resolveCtx),
resolveModel(manifest, 'embedding', resolveCtx)
]);
// Initialize the gateway
const gateway = createGateway({ apiKey: env.AI_GATEWAY_API_KEY });
const checklistScorerModel = gateway(checklistRoutingId);
```
Sources: [apps/web-svelte/src/routes/mee/practice/packet/[packetId]/+page.server.ts:121-131]()
## Benchmarking and Multi-Model Support
The system is designed to facilitate benchmarking across multiple models to evaluate latency, cost, and grading quality. The `benchmark-models.ts` script utilizes the transport infrastructure to run the same submission through various configurations (e.g., `gpt-4o-mini`, `gemini-3-flash`, `mistral-small-3.2`).
### Benchmark Metrics
The system tracks several metrics per model configuration:
* **Latency**: Measured in milliseconds for embedding, quality channel, and adherence calls.
* **Cost**: Estimated in USD based on input/output token pricing per million.
* **Attribution Quality**: Validated via a "Sonnet judge" to ensure evidence is correctly attributed to legal issues.
Sources: [apps/web-svelte/scripts/benchmark-models.ts:122-150](), [apps/web-svelte/scripts/benchmark-models.ts:310-330]()
## Conclusion
The Model Manifest & Transport system provides the necessary abstraction and security guardrails for Bar Prep by SHEP's AI-driven features. By decoupling model selection from business logic and enforcing strict retention and allowlist policies, the system ensures that legal grading remains accurate, compliant, and easy to optimize through structured benchmarking.
### Model Benchmark & Drift Baselines
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/scripts/benchmark-models.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-models.ts)
- [apps/web-svelte/scripts/benchmark-confidence.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-confidence.ts)
- [apps/web-svelte/scripts/ab-test-evidence-attribution.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/ab-test-evidence-attribution.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
# Model Benchmark & Drift Baselines
The Model Benchmark & Drift Baselines system provides a rigorous framework for evaluating Large Language Model (LLM) performance, grading accuracy, and longitudinal stability within the Bar Prep by SHEP platform. It ensures that automated grading across NextGen and MEE exam types remains consistent despite updates to underlying AI models or system prompts.
This system serves as a quality gate, using anchor responses, adversarial cases, and multi-model comparison scripts to measure latency, cost, scoring variance, and evidence attribution quality. By establishing drift baselines, the project can detect when model updates (e.g., a transition from GPT-4o to a newer iteration) fundamentally alter the grading distribution or reliability of feedback.
## System Architecture
The benchmarking system is built upon three primary pillars: standardized benchmark suites, execution scripts for performance measurement, and database-level tracking of drift baselines.
### Benchmark Execution Flow
The following diagram illustrates how the system runs a multi-model benchmark against a standardized submission to generate performance metrics.
```mermaid
graph TD
Data[Load Test Data: Bundle & Essay] --> Config[Load Model Manifests]
Config --> Iter[Iterate Through Models]
subgraph Execution
Iter --> Grade[gradeSubmission]
Grade --> Timing[Capture Latency]
Grade --> Cost[Calculate Est. Cost]
end
subgraph Validation
Grade --> Judge[Sonnet Judge: Evidence Attribution]
Judge --> Relevant[Measure Relevance]
end
Relevant --> Report[Generate Results Table]
```
The benchmark process pulls a specific `ExecutableScoringBundle` and a benchmark submission from the database, runs the grading runtime for each configured model, and then utilizes a high-capability "Judge" model (typically Claude 3.5 Sonnet) to verify the relevance of the evidence extracted by the models.
Sources: [apps/web-svelte/scripts/benchmark-models.ts:162-234](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:39-50]()
## Core Components
### 1. Benchmark Suite Schema
The `BenchmarkSuite` is a Zod-validated structure that defines the expected outcomes for a set of "Anchor" and "Adversarial" cases.
* **Anchors:** High-quality responses with predefined bands (e.g., bands 2, 4, 6) used to calibrate the model.
* **Adversarial Cases:** Intentionally flawed responses (e.g., "verbose rule dump" or "fact summary no law") used to test if the model correctly penalizes poor legal reasoning.
* **Certification Thresholds:** Specific requirements for passing, such as minimum average confidence and maximum band error.
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:109-158]()
### 2. Drift Monitoring
The system tracks "Inter-Model Drift," which represents the variance in scoring between two different model manifests when processing the same bundle.
| Field | Type | Description |
| :--- | :--- | :--- |
| `bundle_id` | TEXT | ID of the scoring bundle being tested. |
| `baseline_gap` | INTEGER | The measured difference in performance or scoring. |
| `manifest_a_id` | TEXT | The control model manifest ID. |
| `manifest_b_id` | TEXT | The treatment/newer model manifest ID. |
Sources: [apps/web-svelte/database/schema.sql:447-455]()
## Benchmarking Methodologies
### Confidence Calibration
The system compares binary scoring against confidence-weighted scoring. This is used to determine if the model’s internal confidence levels should affect the final grade. The benchmark re-scores the same adherence results with both methods to isolate the scoring difference.
```mermaid
sequenceDiagram
participant B as Benchmark Script
participant G as gradeSubmission
participant A as Aggregator
B->>G: Execute Grade with Model
G-->>B: Return Adherence Results
B->>A: Aggregate (Normal Confidence)
A-->>B: Result A
B->>A: Aggregate (Binary/Overridden Confidence)
A-->>B: Result B
Note over B,A: Compare Delta in Raw Score & Band
```
Sources: [apps/web-svelte/scripts/benchmark-confidence.ts:153-188]()
### Evidence Attribution A/B Testing
To improve the accuracy of how models cite student text (evidence), the system supports A/B tests on prompt variants.
* **Control:** Standard adherence prompt.
* **Treatment:** Adherence prompt plus "Negative Context" (providing a list of other issues to prevent cross-attribution).
* **Measurement:** A "Sonnet Judge" evaluates every `EvidenceRecord` for relevance to the attributed issue.
Sources: [apps/web-svelte/scripts/ab-test-evidence-attribution.ts:98-120](), [apps/web-svelte/scripts/ab-test-evidence-attribution.ts:175-220]()
## Data Models
The following table summarizes the key tables used for storing benchmark results and quality gates.
| Table | Purpose | Key Constraints |
| :--- | :--- | :--- |
| `v3_2_benchmark_suites` | Stores the JSON definition of test cases and anchors. | `UNIQUE(benchmark_suite_id, version)` |
| `quality_gate_results` | Summarizes pass/fail status for a packet's certification. | `verification_rate >= 0.9` |
| `certification_reports` | Detailed logs of section-by-section benchmark performance. | `FOREIGN KEY (bundle_id)` |
| `v3_2_regression_runs` | Individual run results for anchors during a bundle update. | `FOREIGN KEY (grading_run_id)` |
Sources: [apps/web-svelte/database/schema.sql:379-400](), [apps/web-svelte/database/schema.sql:637-652](), [apps/web-svelte/database/schema.sql:856-867]()
## Summary
The "Model Benchmark & Drift Baselines" module is a critical infrastructure component for Bar Prep by SHEP. It allows the team to quantitatively prove that the AI grading engine (V3.2) meets professional standards. By combining deterministic verification (evidence substring matching) with LLM-based judging (attribution relevance), the system provides a comprehensive view of model quality, latency, and cost-effectiveness.
Sources: [apps/web-svelte/scripts/benchmark-models.ts:245-280](), [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:190-210]()
### Evidence Verification
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.test.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.test.ts)
- [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/server/v3_2/runtime/index.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts)
- [apps/web-svelte/scripts/ab-test-evidence-attribution.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/ab-test-evidence-attribution.ts)
- [apps/web-svelte/scripts/benchmark-confidence.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-confidence.ts)
# Evidence Verification
Evidence Verification is a deterministic runtime process within the grading engine designed to validate that quotes provided by Large Language Models (LLMs) actually exist within the student's submission. This system serves as a "hallucination check," ensuring that any text cited as evidence for a legal issue is a legitimate excerpt from the essay. Unlike the initial grading steps which rely on LLM inference, evidence verification is strictly algorithmic, using string normalization and substring matching to link claims back to specific spans of text.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:110](), [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:36-40]()
## Architecture and Workflow
The verification process occurs after the "Packet Adherence" channel has identified potential legal issues and extracted supporting quotes. The runtime iterates through every checklist result that contains `quoted_evidence` and attempts to resolve these quotes against the preprocessed version of the student's essay.
### Logic Flow
The system follows a hierarchical matching strategy:
1. **Sentence-Level Match**: The verifier first attempts to find the normalized quote within any single sentence.
2. **Paragraph-Level Fallback**: If no sentence-level match is found and the policy allows, the verifier attempts to find the quote within a broader paragraph context.
3. **Rejection**: If neither scope produces a match, the evidence is marked as `rejected`.
```mermaid
flowchart TD
Start[Start Verification] --> NormQuote[Normalize Quoted Text]
NormQuote --> EmptyCheck{Is Quote Empty?}
EmptyCheck -- Yes --> Reject[Result: Rejected]
EmptyCheck -- No --> SentenceLoop[Iterate Sentences]
SentenceLoop --> SentMatch{Normalized Match?}
SentMatch -- Yes --> VerifiedS[Result: Verified - Sentence Exact]
SentMatch -- No --> MoreSent{More Sentences?}
MoreSent -- Yes --> SentenceLoop
MoreSent -- No --> FallbackEnabled{Fallback Allowed?}
FallbackEnabled -- No --> Reject
FallbackEnabled -- Yes --> ParaLoop[Iterate ParagraphS]
ParaLoop --> ParaMatch{Normalized Match?}
ParaMatch -- Yes --> VerifiedP[Result: Verified - Paragraph Fallback]
ParaMatch -- No --> MorePara{More Paragraphs?}
MorePara -- Yes --> ParaLoop
MorePara -- No --> Reject
```
This diagram illustrates the deterministic search logic used to resolve LLM-provided quotes against student essay text.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:36-92]()
## Normalization Process
To handle minor discrepancies in whitespace, casing, or punctuation between the LLM's extraction and the original text, the verifier applies a series of normalization steps defined in the `EvidenceVerificationPolicy`.
### Normalization Steps
| Step | Implementation Logic |
| :--- | :--- |
| `trim` | Removes leading and trailing whitespace. |
| `collapse_whitespace` | Replaces multiple whitespace characters with a single space. |
| `lowercase` | Converts all characters to lowercase. |
| `strip_punctuation` | Removes all non-alphanumeric characters (using Unicode categories) and collapses resulting spaces. |
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:16-34](), [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts:285-290]()
## Data Structures
The system relies on specific schemas to define verification rules and record outcomes.
### Evidence Verification Policy
This configuration (part of the `ExecutableScoringBundle`) dictates how strictly the verifier behaves.
* **mode**: Currently only `normalized_substring` is implemented.
* **normalization_steps**: An array of steps (e.g., `['trim', 'lowercase']`).
* **fallback_to_paragraph_scope**: Boolean determining if the search expands to paragraphs if sentence matching fails.
Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts:282-291](), [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:41-43]()
### Verification Result
The `VerificationResult` interface captures the output of the verification attempt for a single quote.
```typescript
export interface VerificationResult {
verified: boolean;
verification_mode: 'sentence_exact' | 'paragraph_fallback' | 'rejected';
normalized_quoted_text: string;
resolved_span_id: string | null;
resolved_span_type: 'sentence' | 'paragraph' | null;
}
```
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:6-14]()
## Impact on Scoring
Evidence verification is not just a metadata task; it directly influences the final grade through "Hard Band Rules" and scoring caps.
1. **Verified Counts**: The runtime tallies `verifiedCount` per checklist item and `verifiedCountsByIssue` per legal issue.
2. **Scoring Caps**: If an issue requires evidence (defined in `evidence_rules`) but the verifier fails to validate the quotes, the issue score may be capped (e.g., `max_score_without_evidence`).
3. **Hard Band Rules**: Certain triggers, such as `verified_issue_hit_count`, can enforce a maximum band (e.g., Band 1 or 2) if the number of verified evidence pieces falls below a threshold.
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/index.ts:133-145](), [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts:145-150](), [apps/web-svelte/scripts/benchmark-confidence.ts:223-238]()
## Implementation Details
The core logic is contained in the `verifyEvidence` function. It takes a raw string quote and compares it against `AtomicSpan` objects (sentences and paragraphs) generated during the preprocessing phase.
```typescript
// Example from apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:58-69
for (const sentence of sentences) {
const normalizedSpan = applyNormalization(sentence.text, policy.normalization_steps);
if (normalizedSpan.includes(normalizedQuote)) {
return {
verified: true,
verification_mode: 'sentence_exact',
normalized_quoted_text: normalizedQuote,
resolved_span_id: sentence.span_id,
resolved_span_type: 'sentence'
};
}
}
```
Sources: [apps/web-svelte/src/lib/server/v3_2/runtime/evidence-verifier.ts:58-69]()
Evidence Verification ensures the integrity of the AI-generated feedback by anchoring all legal claims in the actual text written by the student, providing a robust defense against LLM hallucinations.
## Deployment/Infrastructure
### Vercel Deployment & Infrastructure
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/package.json](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/package.json)
- [package.json](https://github.com/airobal/barprep-by-shep/blob/main/package.json)
- [CLAUDE.md](https://github.com/airobal/barprep-by-shep/blob/main/CLAUDE.md)
- [README.md](https://github.com/airobal/barprep-by-shep/blob/main/README.md)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
- [HANDOFF.md](https://github.com/airobal/barprep-by-shep/blob/main/HANDOFF.md)
# Vercel Deployment & Infrastructure
The Vercel deployment and infrastructure for Bar Prep by SHEP provides a scalable, high-performance environment for hosting a modern SvelteKit application. The project is structured as a monorepo utilizing npm workspaces to manage the primary SvelteKit web application and a content ingestion pipeline. The infrastructure is designed to support high-stakes bar exam preparation for both the NextGen UBE and MEE formats.
The core of the deployment strategy relies on the Vercel adapter for SvelteKit, which allows the application to leverage Vercel's edge network, serverless functions, and global distribution. This setup ensures that the application can handle the computational demands of legal reasoning practice while maintaining low latency for users across different jurisdictions.
Sources: [apps/web-svelte/package.json:44](), [README.md:10-15](), [CLAUDE.md:15-20]()
## Monorepo Architecture
The project is organized into a monorepo structure using `npm workspaces`. This allows for shared dependencies and consistent tooling across the web application and supporting packages.
```mermaid
graph TD
Root[Project Root] --> Apps[apps/]
Root --> Pkgs[packages/]
Apps --> WebSvelte[web-svelte]
Pkgs --> ContentPipeline[content-pipeline]
WebSvelte --> SvelteKit[SvelteKit 2]
WebSvelte --> VercelAdapter[Adapter Vercel]
```
The diagram above illustrates the monorepo hierarchy, where the SvelteKit application resides in `apps/web-svelte` and is prepared for Vercel deployment via a dedicated adapter.
### Workspace Configuration
The root `package.json` defines the workspaces used for managing the repository structure.
| Workspace Path | Description |
| :--- | :--- |
| `apps/*` | Contains the primary SvelteKit web application (`web-svelte`). |
| `packages/*` | Contains supporting packages like the `content-pipeline`. |
Sources: [package.json:16-19](), [CLAUDE.md:15-18]()
## Tech Stack & Build Pipeline
The deployment infrastructure is built around a modern TypeScript stack, utilizing Svelte 5 (runes) and Tailwind CSS 4. The build process is orchestrated via Vite.
### Core Frameworks
- **Frontend Framework:** SvelteKit 2 with Svelte 5 (runes).
- **Styling:** Tailwind CSS 4 using the `@tailwindcss/vite` plugin.
- **Runtime:** Node.js >= 20.0.0.
- **Bundler:** Vite 5.
Sources: [apps/web-svelte/package.json:34-48](), [package.json:23-25](), [CLAUDE.md:23-28]()
### Build Commands
Deployment on Vercel triggers the standard npm build scripts defined in the workspace.
```mermaid
flowchart TD
Start[Vercel Trigger] --> Install[npm install]
Install --> Sync[svelte-kit sync]
Sync --> Build[vite build]
Build --> Adapt[Adapter Vercel Processing]
Adapt --> Deploy[Serverless/Edge Deployment]
```
This flowchart shows the progression from a Vercel trigger to the final deployment of serverless assets.
| Command | Script | Description |
| :--- | :--- | :--- |
| Build | `npm run build` | Executes `vite build` within the `apps/web-svelte` workspace. |
| Check | `npm run check` | Runs `svelte-check` to ensure type safety before deployment. |
| Migrate | `npm run migrate` | Executes database migrations against the Neon PostgreSQL instance. |
Sources: [apps/web-svelte/package.json:7-12](), [package.json:6-14]()
## Database & Persistence Infrastructure
Bar Prep by SHEP utilizes a dual-database architecture hosted on Neon PostgreSQL. This infrastructure design separates user authentication and core "SHEP" data from the specific bar preparation exam data.
### Dual-Database Connectivity
The application connects to two distinct databases within the same Neon project:
1. **`DATABASE_URL`:** Dedicated to barprep exam data (sessions, tasks, drafts, submissions).
2. **`AUTH_DATABASE_URL`:** Connects to the SHEP core database for shared user management and Auth.js integration.
```mermaid
flowchart TD
App[web-svelte] --> AuthDB[(SHEP Core Auth DB)]
App --> PrepDB[(Barprep Exam DB)]
AuthDB --- UserTable[Users Table]
PrepDB --- ExamData[Sessions/Tasks/Drafts]
UserTable -.->|Cross-DB Link| ExamData
```
The diagram represents the separation between the authentication database and the exam-specific database.
### Infrastructure Components
| Component | Provider/Tool | Role |
| :--- | :--- | :--- |
| Database | Neon PostgreSQL | Managed serverless PostgreSQL hosting. |
| Authentication | Auth.js (`@auth/sveltekit`) | Handles session management and database adapters. |
| DB Driver | `postgres` / `pg` | Used for direct queries and Auth.js adapters. |
Sources: [CLAUDE.md:46-60](), [PRODUCT_SPEC.md:105-110](), [apps/web-svelte/package.json:52-57]()
## Environment Configuration
The deployment environment requires several critical variables to be configured in Vercel to ensure connectivity to the serverless infrastructure and database.
### Required Environment Variables
- `BARPREP_DATABASE_URL`: Connection string for the exam database.
- `AUTH_DATABASE_URL`: Connection string for the shared SHEP core user database.
- `AI_GATEWAY_API_KEY`: Required for components utilizing the `@ai-sdk/gateway` (used for legal reasoning analysis).
- `STRIPE_API_KEY`: Reserved for future payment integration ($299 per cycle).
Sources: [README.md:10-15](), [CLAUDE.md:47-50](), [HANDOFF.md:43-50]()
## Summary
The Vercel infrastructure for Bar Prep by SHEP is designed for high reliability and separation of concerns. By leveraging the SvelteKit Vercel adapter and Neon's serverless PostgreSQL, the system provides a robust foundation for legal education. The dual-database strategy allows the project to scale independently while maintaining integration with the broader SHEP ecosystem.
### Tooling, Scripts & Seeding
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/scripts/seed-all-packets.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/seed-all-packets.ts)
- [apps/web-svelte/scripts/benchmark-confidence.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/scripts/benchmark-confidence.ts)
- [apps/web-svelte/database/migrations/002_seed_data.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/002_seed_data.sql)
- [package.json](https://github.com/airobal/barprep-by-shep/blob/main/package.json)
- [README.md](https://github.com/airobal/barprep-by-shep/blob/main/README.md)
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
# Tooling, Scripts & Seeding
Tooling, scripts, and seeding mechanisms in the Bar Prep by SHEP project provide the foundation for environment setup, database management, and content ingestion. These utilities automate the creation of the database schema, populate the system with both deterministic development data and complex licensed exam content, and provide benchmarking tools for the grading engine.
The system relies on a dual-database architecture where barprep-specific exam data is isolated from the core SHEP authentication data. Management of these resources is handled via npm workspace scripts and dedicated TypeScript execution utilities (tsx).
Sources: [CLAUDE.md](), [README.md:1-25](), [package.json:6-16]()
## Database Management & Migrations
The project uses a structured migration system to manage its PostgreSQL schema. Migrations are stored as sequential SQL files in the `apps/web-svelte/database/migrations/` directory.
### Migration Workflow
The lifecycle of database schema management involves applying forward migrations and generating canonical snapshots.
| Command | Description | Environment Variable |
|---------|-------------|----------------------|
| `npm run migrate` | Applies forward SQL migrations to the target database. | `BARPREP_DATABASE_URL` |
| `npm run migrate:snapshot` | Regenerates the `schema.sql` snapshot from a scratch database or Docker container. | `BARPREP_SCRATCH_DATABASE_URL` |
Sources: [README.md:12-25](), [package.json:14-15]()
### Schema Initialization
The initial schema (Migration 001) establishes shared tables for users, sessions, and drafts, alongside domain-specific tables for NextGen tasks and MEE essays.
Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:1-15]()
```mermaid
flowchart TD
A[Migration Scripts] --> B{Action?}
B -- Apply --> C[Forward Migrations]
B -- Snapshot --> D[Scratch DB / Docker]
C --> E[(Production/Dev DB)]
D --> F[Generate schema.sql]
F --> G[Canonical Reference]
```
This diagram shows the relationship between migration scripts and the resulting database state or schema documentation.
Sources: [README.md:20-25]()
## Content Seeding
Seeding is divided into two categories: deterministic development seeding for local QA and complex content seeding for the V3.2 grading engine.
### Deterministic Development Seeding
Migration `002_seed_data.sql` populates the database with stable, fixed-UUID records to ensure routes and fixtures remain consistent across developer environments. This includes:
* **NextGen Tasks:** Example "Green Industries Contract Dispute" (SPT).
* **NextGen Legal Materials:** Restatements and UCC statutes.
* **MEE Essays:** Business Associations scenario regarding partnership dissolution.
Sources: [apps/web-svelte/database/migrations/002_seed_data.sql:1-12]()
### MEE V3.2 Content Ingestion
The script `seed-all-packets.ts` handles the ingestion of 5 M3 content packets (Contracts, Torts, Evidence, Civil Procedure, and Constitutional Law). This script uses the actual V3.2 compiler to produce bundles identical to production output.
**Key Seeding Steps:**
1. **Model Manifests:** Ensures the presence of specific AI model configurations (e.g., `google-gemini-3.1-flash-lite`).
2. **Retention Posture Checks:** Inserts verification records for data retention modes (e.g., `zero_retention`).
3. **Packet Versioning:** Reads JSON definitions for canonical packets, rubrics, and benchmark suites to publish bundle versions.
Sources: [apps/web-svelte/scripts/seed-all-packets.ts:10-25, 115-155]()
## Benchmarking & Evaluation
The project includes advanced scripts for evaluating the performance and accuracy of the AI-driven grading engine.
### Confidence Calibration Benchmark
The `benchmark-confidence.ts` utility compares binary scoring against confidence-weighted scoring. It isolates the impact of AI model confidence values on final grade bands and component scores.
Sources: [apps/web-svelte/scripts/benchmark-confidence.ts:1-12]()
### Evaluation Flow
The script executes a full grading pipeline once using the standard `gradeSubmission` runtime, then re-scores the adherence results using different aggregation logic.
```mermaid
sequenceDiagram
participant S as Benchmark Script
participant G as Grade Runtime
participant LLM as AI Model (Gemini)
participant A as Aggregator
S->>G: run gradeSubmission()
G->>LLM: Quality & Adherence Calls
LLM-->>G: Results with Confidence
G->>S: Captured Results
S->>A: Aggregate (Confidence-Weighted)
A-->>S: Weighted Score
S->>A: Aggregate (Binary/Forced 1.0)
A-->>S: Binary Score
S->>S: Compare Delta & Band impact
```
This diagram illustrates the sequence for comparing scoring methods during benchmarking.
Sources: [apps/web-svelte/scripts/benchmark-confidence.ts:98-150]()
## Configuration & Environment
Scripts rely on local environment variables, typically loaded from a `.env` file at the application root.
| Variable | Usage |
|----------|-------|
| `BARPREP_DATABASE_URL` | Primary target for migrations and seeding. |
| `BARPREP_SCRATCH_DATABASE_URL` | Temporary database used for schema snapshot generation. |
| `AI_GATEWAY_API_KEY` | Required for benchmarking scripts to communicate with LLM providers. |
Sources: [README.md:15-20](), [apps/web-svelte/scripts/seed-all-packets.ts:40-45](), [apps/web-svelte/scripts/benchmark-confidence.ts:45-50]()
The project uses `npx tsx` with custom TSConfig files (e.g., `scripts/tsconfig.seed.json`) to execute TypeScript scripts directly without a separate build step, ensuring that library code and schemas can be imported directly into maintenance utilities.
Sources: [apps/web-svelte/scripts/seed-all-packets.ts:1-10](), [apps/web-svelte/scripts/benchmark-confidence.ts:1-5]()
## Conclusion
The tooling and seeding infrastructure of Bar Prep by SHEP ensures that the complex legal content and AI grading logic are supported by a stable and reproducible data environment. By separating deterministic development seeds from compiled V3.2 packet versions, the system maintains agility for developers while ensuring high-fidelity testing for the final production grading engine.
## Extensibility and Customization
### Extending the Content Bank
Relevant source files
The following files were used as context for generating this wiki page:
- [apps/web-svelte/database/migrations/001_initial_schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/001_initial_schema.sql)
- [apps/web-svelte/database/migrations/002_seed_data.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/migrations/002_seed_data.sql)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts)
- [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts)
- [apps/web-svelte/database/schema.sql](https://github.com/airobal/barprep-by-shep/blob/main/apps/web-svelte/database/schema.sql)
- [PRODUCT_SPEC.md](https://github.com/airobal/barprep-by-shep/blob/main/PRODUCT_SPEC.md)
# Extending the Content Bank
Extending the Content Bank involves the ingestion and configuration of legal practice materials for two primary domains: the NextGen Bar Exam (Standard Performance Tasks, Legal Research Performance Tasks, and Integrated Question Sets) and the Multistate Essay Examination (MEE). The system is designed to handle both original practice content ("SHEP_PRACTICE") and licensed content from the NCBE ("NCBE_LICENSED").
The content architecture relies on a structured pipeline that transforms raw legal materials into "Canonical Packets" and "Executable Scoring Bundles." This ensures that every piece of content is not only a prompt for the user but also a machine-readable set of rubrics, facts, and legal rules used by the AI scoring engine to evaluate student submissions.
## Data Architecture and Storage
The content bank is underpinned by a PostgreSQL schema that separates metadata and raw content from the highly structured JSON objects required for automated grading. Content is primarily divided into NextGen tasks and MEE essays, each supported by a set of relational tables.
### NextGen Content Schema
NextGen content is centered around `nextgen_tasks`, which represent complex performance scenarios. These tasks link to a library of `nextgen_legal_materials` (statutes, cases, etc.) through a junction table that manages "disclosure steps" for progressive information release.
```mermaid
erDiagram
NEXTGEN_TASKS ||--o{ NEXTGEN_TASK_MATERIALS : contains
NEXTGEN_LEGAL_MATERIALS ||--o{ NEXTGEN_TASK_MATERIALS : referenced_by
NEXTGEN_TASKS {
uuid id
text title
nextgen_task_type task_type
jsonb supervisor_memo
jsonb disclosure_steps
}
NEXTGEN_LEGAL_MATERIALS {
uuid id
text content
nextgen_material_type material_type
text citation
}
NEXTGEN_TASK_MATERIALS {
uuid task_id
uuid material_id
int sort_order
int disclosure_step
text material_key
}
```
*The relationship between NextGen tasks and the shared materials library. Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:102-143]()*
### MEE Content Schema
MEE content focuses on essays and their corresponding model answers. Unlike NextGen, MEE prompts typically lack an external library of materials, focusing instead on fact patterns and "Call of the Question" prompts.
| Table | Purpose | Key Fields |
| :--- | :--- | :--- |
| `mee_essays` | Stores the primary essay prompt and fact pattern. | `fact_pattern`, `call_of_the_question`, `subject` |
| `mee_model_answers` | Contains official or licensed answers used for comparison. | `irac_structure`, `issue_breakdown`, `content` |
| `mee_subject` | Enum defining the 13 core legal subjects (e.g., TORTS, EVIDENCE). | N/A |
*Sources: [apps/web-svelte/database/migrations/001_initial_schema.sql:154-181](), [apps/web-svelte/database/schema.sql:130-144]()*
## Content Ingestion Formats
Adding content requires adherence to strict Zod-validated schemas. The "Canonical Packet" is the primary interchange format for MEE content, encapsulating the prompt, the rubric, and the evidence required for scoring.
### Canonical Packet Structure
A `CanonicalPacket` must define its `schema_profile` (e.g., `llm_only_launch`) and include a complete breakdown of legal issues. Every issue within a packet must have:
* **Rule Atoms:** Atomic units of legal rules.
* **Fact Refs:** References to specific facts in the prompt.
* **Acceptable Paths:** Logical connections between facts and rules that lead to valid conclusions.
* **Evidence Policy:** Requirements for the number of verified quotes needed for various score strengths.
*Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/canonical-packet.ts:1-125]()*
### Executable Scoring Bundles
While the Canonical Packet contains the "what," the `ExecutableScoringBundle` contains the "how" for the grading engine. It includes:
1. **Segmentation Policy:** How to split student text (sentences vs. paragraphs).
2. **Retrieval Policy:** Embedding models and similarity thresholds for finding relevant student text.
3. **Hard Band Rules:** Deterministic triggers (like word count thresholds) that cap the maximum possible score regardless of content quality.
```mermaid
flowchart TD
Raw[Raw Content/PDF] --> Norm[Normalization Phase]
Norm --> Bund[Issue Bundle Derivation]
Bund --> Agg[Aggregation Policy Emission]
Agg --> Final[Executable Scoring Bundle]
Final --> DB[(PostgreSQL)]
```
*The compilation flow from raw content to a machine-executable scoring bundle. Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/executable-scoring-bundle.ts:252-290](), [apps/web-svelte/src/lib/server/v3_2/compiler/phases.ts:1-20]()*
## Implementation Example: Seed Data
To extend the bank locally for development, entries can be added to the migration scripts using fixed UUIDs to ensure route stability.
```sql
INSERT INTO nextgen_tasks (
id, title, task_type, skills_group, foundational_concepts, expected_time_minutes, role_prompt
) VALUES (
'a1b2c3d4-0001-4000-8000-000000000001',
'Green Industries Contract Dispute',
'SPT', 'A',
ARRAY['Contracts', 'Civil Procedure'],
90,
'You are a junior associate... prepare a memorandum analyzing the breach of contract claim.'
);
```
*Sources: [apps/web-svelte/database/migrations/002_seed_data.sql:22-35]()*
## Quality and Benchmarking
New content must be certified against a `BenchmarkSuite` to ensure scoring accuracy. This suite contains "Anchors"—exemplar student responses mapped to specific performance bands (1–6).
* **Absolute Cases:** Responses with expected score ranges and issue findings.
* **Adversarial Cases:** Responses designed to test the system (e.g., "verbose_rule_dump" or "legalese_off_point").
* **Self-Consistency:** The system must produce similar results for the same input across multiple runs.
*Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:1-135]()*
### Certification Thresholds
| Metric | Description | Minimum Requirement |
| :--- | :--- | :--- |
| `min_absolute_case_pass_rate` | Percentage of benchmark cases that must hit the expected band. | Specified per suite |
| `max_band_error` | Maximum allowable difference between expected and actual band. | Typically 0 or 1 |
| `min_average_confidence` | The average LLM confidence score required for certification. | User-defined |
*Sources: [apps/web-svelte/src/lib/mee/v3_2/schemas/benchmark-suite.ts:111-122]()*
## Summary
Extending the Content Bank is a multi-step process involving relational database entry for metadata, and the compilation of Canonical Packets into Executable Scoring Bundles for the AI grading engine. Each new task or essay must be validated against a Benchmark Suite containing both standard and adversarial cases to ensure scoring reliability and adherence to NCBE standards.