# Product Spec: SHEP Runtime Bar Essay Grading Engine Status: draft v0.1 Purpose: define the product, technical architecture, legal-risk posture, and marketing boundaries for a bar essay grading workflow that supports original SHEP prompts and user-uploaded practice prompts. Not legal advice. This is a product-risk specification for counsel and engineering review. --- ## 1. Executive Decision SHEP should build a **runtime, user-directed essay grading engine** that generates grading packets from student-provided prompts asynchronously while the student reads, outlines, and writes. The recommended launch model is: **SHEP Original Mode** SHEP provides original simulated bar essay prompts and certified SHEP rubrics. These can be fully stored, reused, benchmarked, and marketed as SHEP-created practice content. **Upload Authorized Prompt Mode** The student uploads or pastes a practice prompt they are authorized to use. SHEP generates a session-specific grading packet in the background. The packet is used only for that student’s grading session. By default, SHEP does not persist the prompt or prompt-derived packet on the backend. **Generic Certified Rubric Catalog** SHEP may maintain reusable, certified rubrics for generic doctrine and issue archetypes: offer, consideration, hearsay, personal jurisdiction, negligence, secured transactions attachment, etc. This is different from maintaining a database of packets keyed to real NCBE MEE questions. **Do not launch a central real-MEE packet bank without a license.** NCBE identifies examinations, study aids, sample questions, model analyses, and point sheets as protected content and restricts reproduction, modification, distribution, copying, and derivative works. NCBE also requires written consent for use of marks such as NCBE, MEE, MPT, MBE, and UBE. ([National Conference of Bar Examiners][1]) --- ## 2. Product Goal SHEP helps bar candidates practice essay writing by giving fast feedback on: Issue spotting Rule accuracy Fact application IRAC organization Conclusion strength Timing and revision strategy The product should be marketed as an **AI-powered bar essay practice and feedback system**, not as an official question bank or official grading substitute. The MEE format supports the async approach because NCBE describes the MEE as six 30-minute questions. The student will naturally spend several minutes reading, outlining, and beginning the response before submitting, which creates enough time for background packet generation. ([NCBE][2]) --- # Part A — Product Modes ## A1. SHEP Original Mode Description: Student selects a SHEP-created simulated prompt. Backend may store: Prompt text Question metadata Certified rubric packet Benchmark data Expected issue map Scoring calibration Model evaluations Student answers Feedback history Legal posture: lowest-risk mode. SHEP owns or controls the prompt and rubric. Product copy: “Practice with original simulated bar essay prompts.” “Get structured feedback on issue spotting, rules, analysis, and organization.” “Built around bar-tested skills and commonly tested subjects.” Avoid saying “official,” “real MEE,” “NCBE-calibrated,” or “past MEE question bank.” --- ## A2. Upload Authorized Prompt Mode Description: Student uploads or pastes a prompt they already have and represents they are authorized to use. Flow: ```text Student uploads prompt ↓ SHEP creates private grading session ↓ Async packet generation starts ↓ Student reads, outlines, and writes ↓ Student submits answer ↓ SHEP grades answer using: (1) generic certified doctrine rubrics (2) session-specific prompt packet ↓ SHEP stores limited feedback ↓ Prompt + prompt-derived packet are deleted or stored locally only ``` Legal posture: moderate but defensible if implemented as user-directed, private, transient processing. Product copy: “Upload a practice prompt you are authorized to use.” “Paste your prompt, write your answer, and receive structured feedback.” “Do not upload unreleased exam materials, confidential materials, or content you are not authorized to process through SHEP.” --- ## A3. California Practice Mode Description: SHEP supports California-style essay practice. Recommended launch implementation: Use SHEP-created California-style simulated prompts. Provide link-outs to the State Bar of California’s official Past Exams page rather than hosting State Bar questions or selected answers. Allow students to paste/upload California materials they are authorized to use. The State Bar of California publishes past exam materials, including examination questions, essay questions and selected answers, and performance tests and selected answers. ([The State Bar of California][3]) But the State Bar also states that its website, web content, and publications are copyrighted by the State Bar of California. ([The State Bar of California][4]) Its linking policy permits links, but says linking sites must not replicate State Bar content, must not imply endorsement, and must state that the link accesses copyrighted State Bar content. ([The State Bar of California][5]) Product copy: “Practice California-style bar essays with original prompts and structured feedback.” “Official State Bar of California past-exam materials are available from the State Bar’s website. Those materials are copyrighted by the State Bar of California. SHEP is independent and is not affiliated with or endorsed by the State Bar.” Avoid: “Our California past essay library.” “Complete California essay bank.” “State Bar selected answers included.” --- ## A4. Optional Local Save Mode Description: Student may save the generated packet locally in the browser. Preferred storage: IndexedDB for structured browser-side storage. IndexedDB is designed for client-side storage of significant amounts of structured data, including files/blobs. ([MDN Web Docs][6]) Flow: ```text Backend generates packet ↓ Backend returns packet to browser ↓ Backend deletes packet ↓ Student optionally saves locally ↓ Packet persists only in browser storage ``` Product copy: “Save this study packet locally in your browser.” “Locally saved packets are stored on your device/browser and are not added to SHEP’s central rubric library.” Only use this copy if the architecture actually implements local-only storage. --- # Part B — Technical Specification ## B1. Recommended Architecture The core architecture should be: ```text Generic certified rubric catalog + Runtime session packet generation + Limited stored student performance data ``` Do not build: ```text Central database of real-MEE-specific answer packets ``` The critical distinction is between **generic issue archetypes** and **specific copyrighted prompts**. Safe/reusable: ```json { "rubric_id": "contracts_consideration_v2", "subject": "Contracts", "issue": "consideration", "rule_elements": [ "a bargained-for exchange is required", "legal detriment or benefit may satisfy consideration", "past consideration is generally insufficient" ], "grading_dimensions": [ "issue_spotting", "rule_accuracy", "fact_application", "conclusion" ] } ``` Higher-risk/session-scoped: ```json { "session_packet_id": "sesspkt_123", "source": "user_uploaded_prompt", "subject": "Contracts", "detected_issues": [ "formation", "consideration", "breach" ], "fact_hooks": [ { "issue": "consideration", "abstract_fact": "one party promised performance in exchange for a disputed legal right" } ] } ``` --- ## B2. Runtime Job Pipeline ### Job 1 — Intake and Attestation Triggered when the student uploads or pastes a prompt. Required UI attestation: “By uploading this prompt, I confirm I have the right to use it for private study and to process it through SHEP. I will not upload unreleased exam materials, confidential materials, or content I am not authorized to use.” Create session object: ```json { "session_id": "sess_123", "mode": "user_uploaded_prompt", "user_attested_authorized_use": true, "prompt_storage_policy": "transient", "packet_storage_policy": "transient_or_local_only", "created_at": "2026-04-24T12:00:00Z" } ``` --- ### Job 2 — Fast Classifier Runs immediately after prompt intake. Outputs: Subject Calls Likely issue archetypes Confidence score Whether packet generation is possible Example: ```json { "session_id": "sess_123", "subject": "Contracts", "confidence": 0.91, "detected_calls": [ "Analyze whether Party A has a claim against Party B." ], "likely_issue_archetypes": [ "formation", "consideration", "assignment", "breach" ] } ``` UI message: “Detected: Contracts. Likely issues: formation, consideration, assignment, breach. Start writing; SHEP is preparing the grading framework.” --- ### Job 3 — Generic Rubric Retrieval Pull from SHEP-owned doctrine catalog. Example: ```json { "session_id": "sess_123", "rubrics": [ "contracts_formation_v2", "contracts_consideration_v2", "contracts_assignment_v1", "contracts_breach_v2" ] } ``` This catalog can be tested, versioned, benchmarked, and stored persistently. --- ### Job 4 — Session Packet Generation Generate prompt-specific fact hooks and expected analysis structure. Storage policy: transient by default. Example: ```json { "session_packet_id": "sesspkt_123", "session_id": "sess_123", "rubric_basis": [ "contracts_formation_v2", "contracts_consideration_v2", "contracts_breach_v2" ], "fact_hooks": [ { "rubric_id": "contracts_consideration_v2", "abstract_fact": "the exchange involved a disputed claim or legal entitlement" }, { "rubric_id": "contracts_breach_v2", "abstract_fact": "one party later refused to perform" } ], "backend_persisted": false, "browser_local_save_available": true, "expires_after_minutes": 90 } ``` Design requirement: fact hooks should be abstracted, not copied as close paraphrases of the original prompt. --- ### Job 5 — Answer Grading Triggered when student submits the answer. Inputs: Student answer Generic doctrine rubrics Session packet Optional timing data Optional student outline if captured Outputs: Score band Issue-by-issue feedback Rule feedback Application feedback Organization feedback Revision plan Persist this: ```json { "submission_id": "sub_123", "user_id": "user_456", "mode": "user_uploaded_prompt", "stored_prompt": false, "stored_prompt_packet": false, "subject": "Contracts", "issue_tags": [ "formation", "consideration", "breach" ], "score_band": 4, "feedback": { "issue_spotting": "You identified the central contract issue but missed one secondary issue.", "rule_accuracy": "Your rule statement needed more precision.", "application": "Your answer needed tighter connection between the facts and the legal standard.", "organization": "The answer would improve with clearer IRAC separation." } } ``` Do not persist this by default: ```json { "source": "July 2021 MEE Contracts", "key_facts": [ "Painter agreed to paint Neighbor's house", "Neighbor promised to assign a claim", "Painter refused after learning the claim was disputed" ], "expected_conclusion": "Neighbor likely succeeds", "point_weights": { "formation": 20, "assignment": 30, "breach": 25 } } ``` --- ## B3. Storage Rules | Artifact | Store? | Location | Notes | | ---------------------------------- | ------------: | -------------------------- | -------------------------------------------------- | | SHEP original prompt | Yes | Backend | SHEP-owned content. | | SHEP original certified rubric | Yes | Backend | Persistent, reusable. | | Generic doctrine rubric | Yes | Backend | Persistent, reusable. | | User-uploaded prompt | No by default | Transient only | Delete after packet/grading workflow. | | Prompt-derived session packet | No by default | Transient or browser-local | Avoid central storage. | | Student answer | Yes | Backend | Private to user account. | | Feedback summary | Yes | Backend | Avoid close prompt paraphrases. | | Score/performance analytics | Yes | Backend | Store issue tags and skill metrics. | | Prompt embeddings | No | None | Avoid vectorizing user-uploaded protected prompts. | | Model traces/logs with prompt text | No | None | Disable or scrub. | | Error logs containing prompt text | No | None | Redact request bodies. | --- ## B4. Provider Retention Controls The product’s privacy copy must match the actual model-provider setup. OpenAI states that API data is not used to train models by default, but abuse-monitoring logs may contain prompts/responses and are retained up to 30 days unless a customer is approved for Modified Abuse Monitoring or Zero Data Retention. OpenAI also lists endpoint-specific retention behavior, including that Responses background mode stores response data to disk for roughly 10 minutes to enable polling. ([OpenAI Developers][7]) Anthropic states that its Claude API offers Zero Data Retention arrangements where customer data is not stored at rest after the API response returns, except as needed for law or misuse prevention; it also states that ZDR eligibility varies by feature. ([Claude][8]) Technical requirement: Do not claim “we do not store prompts” unless all of the following are true: SHEP backend does not store prompts. Background jobs do not persist prompts beyond the TTL. Provider retention settings are compatible with the claim. Tracing tools do not store prompts. Sentry/PostHog/OpenTelemetry/LangSmith/Vercel logs do not capture prompts. Vector stores do not contain prompts or prompt-derived embeddings. --- ## B5. Logging and Observability Requirements Prompt text must be redacted from: Request logs Response logs Background job payloads Queue dead-letter payloads Tracing spans LLM prompt traces Error reports Analytics events Database query logs Support/admin dashboards Allowed analytics fields: ```json { "subject": "Contracts", "mode": "user_uploaded_prompt", "issue_tags": ["formation", "breach"], "packet_generation_ms": 18000, "grading_ms": 9000, "score_band": 4 } ``` Disallowed analytics fields: ```json { "prompt_text": "...", "key_facts": ["..."], "full_packet": "...", "quoted_call": "..." } ``` --- ## B6. Fallback Behavior If packet generation finishes before submission: Grade using generic rubrics + session packet. If packet generation is delayed: Grade first using generic subject rubric, then optionally refine when packet completes. If classifier confidence is low: Ask the user to select subject or confirm detected issue category. If user submits immediately: Use a generic rubric and disclose: “Because the prompt-specific framework was not ready, this feedback is based on the detected subject and general issue structure.” --- ## B7. Quality Controls For SHEP Original Mode: Benchmark against internal gold answers. Store calibration data. Version rubrics. Run regression tests. For Upload Authorized Prompt Mode: Do not call output “certified.” Label it as “adaptive study feedback.” Track confidence. Show uncertainty when issue detection is weak. Use generic doctrine rubrics as the stable grading foundation. Recommended label: “Generated feedback framework” Avoid label: “Official rubric” “Answer key” “NCBE packet” “Real MEE scoring” --- # Part C — Legal and Compliance Concerns ## C1. Copyright Baseline Legal rules, procedures, systems, concepts, and principles are not protected by copyright under 17 U.S.C. § 102(b). This supports building generic doctrine rubrics around legal concepts like offer, consideration, hearsay, negligence, jurisdiction, and remedies. ([Legal Information Institute][9]) But copyright owners have exclusive rights to reproduce copyrighted works and prepare derivative works based on them under 17 U.S.C. § 106. ([Legal Information Institute][10]) That is why real-question-specific packets containing close fact maps, expected conclusions, and point weights can create risk if based on NCBE or State Bar materials. --- ## C2. NCBE / MEE Risk NCBE risk is high for any platform-owned bank of real MEE prompts, model analyses, point sheets, or question-specific packets. NCBE says its protected content includes examinations, study aids, sample questions, model analyses, point sheets, and other text. It prohibits selling, modifying, reproducing, displaying, distributing, sharing, transferring, copying, or creating derivative works from content without permission. ([National Conference of Bar Examiners][1]) NCBE also sells official study aids, including legacy MEE/MPT materials, and describes those materials as authentic, official materials created by bar exam authors. ([NCBE][11]) That increases risk if SHEP’s product looks like an unlicensed substitute for official NCBE study aids. Legal product rule: Do not store or market a master database of real-MEE-specific packets without a license. --- ## C3. California Risk California is more favorable than NCBE because the State Bar publishes past exams. But public access does not equal unrestricted commercial reuse. The State Bar’s Past Exams page lists examination questions, essay questions and selected answers, and performance tests and selected answers. ([The State Bar of California][3]) However, the State Bar’s policies state that its website, web content, and publications are copyrighted. ([The State Bar of California][4]) Its linking policy says websites may link if they comply with conditions, including not replicating State Bar content and not implying endorsement. ([The State Bar of California][5]) Legal product rule: For California, prefer original prompts plus official-source link-outs. Do not scrape, replicate, or host State Bar questions/selected answers without permission or legal clearance. --- ## C4. DMCA / User-Directed Storage If SHEP stores user-uploaded or user-derived material on its servers, it should implement a DMCA policy. Section 512 includes a limitation for “storage at the direction of a user,” but the service provider must satisfy conditions such as lack of disqualifying knowledge, expeditious removal/disablement after notice, and designation of an agent. ([Legal Information Institute][12]) Legal product rule: If backend user-private storage is offered, implement: DMCA agent registration Public takedown policy Repeat-infringer policy User counter-notice process Expeditious takedown workflow Admin tooling to disable/delete challenged packets --- ## C5. Marketing and Advertising Substantiation The FTC says advertisers need proof before running ads and must have a reasonable basis for objective claims. ([Federal Trade Commission][13]) The FTC’s substantiation policy also covers express and implied objective claims about a product or service. ([Federal Trade Commission][14]) Legal product rule: Do not make claims like “raises your score,” “human-grade accuracy,” “calibrated to real graders,” or “pass with SHEP” unless SHEP has reliable substantiation. Allowed: “Designed to provide structured study feedback.” “Helps identify missed issues and weak rule statements.” “Provides IRAC-focused revision guidance.” Avoid unless substantiated: “Raises your bar essay score.” “Grades like a real bar examiner.” “Human-level accuracy.” “Pass the bar with SHEP.” “Calibrated to official NCBE scoring.” --- # Part D — Marketing Copy Boundaries ## D1. Approved Product Positioning Use this as the master posture: **SHEP helps students practice UBE-style and California-style bar essays with original simulated prompts, user-provided authorized materials, and structured AI feedback on issue spotting, rule accuracy, fact application, and organization.** This says what the product does without claiming official content, official affiliation, or official grading. --- ## D2. Approved Phrases “UBE-style essay practice” “California-style essay practice” “Original simulated bar essay prompts” “Upload a practice prompt you are authorized to use” “Structured feedback on issue spotting, rules, application, and organization” “IRAC-first essay review” “Track missed issues and weak rule statements” “Practice with your own authorized materials” “Independent bar essay study tool” “Not affiliated with or endorsed by NCBE, the State Bar of California, or any board of bar examiners” --- ## D3. Prohibited or High-Risk Phrases “Official MEE grader” “Real MEE question library” “Past MEE question bank” “NCBE-calibrated” “Official NCBE scoring” “Follows official point sheets” “Grades like the NCBE” “Authentic MEE questions included” “Complete MEE library” “State Bar selected answers included” “Human-equivalent grading” “Guaranteed score improvement” “Pass the bar with AI” --- ## D4. Suggested Homepage Copy Headline: **Practice bar essays the way they are evaluated.** Subheadline: **SHEP gives fast, structured feedback on UBE-style and California-style essay answers, helping you spot missed issues, sharpen rules, and improve your analysis before exam day.** Trust line: **Original simulated prompts. User-provided authorized prompt support. No official bar-exam content required.** Feature card: **Original simulated essays** Practice with SHEP-created prompts designed around commonly tested legal skills and subjects. Feature card: **Upload your own authorized prompt** Already have a practice question from a course, tutor, professor, or official source? Paste or upload materials you are authorized to use and get feedback on your answer. Feature card: **IRAC-first feedback** Review issue spotting, rule precision, fact application, organization, and conclusion strength. Disclaimer: **SHEP is an independent study tool. It is not affiliated with, endorsed by, or sponsored by NCBE, the State Bar of California, or any board of bar examiners.** --- # Part E — Risk Matrix | Feature | Risk | Ship? | Notes | | ---------------------------------------------- | ------------: | ------------: | ------------------------------------------ | | SHEP original prompts + SHEP rubrics | Low | Yes | Best primary product. | | Generic issue-archetype rubric catalog | Low | Yes | Legal doctrine/concepts are safer. | | Runtime packet from user-uploaded prompt | Moderate | Yes | Use transient/session-only design. | | Browser-local packet save | Low–Moderate | Yes | Stronger than backend storage. | | User-private backend packet save | Moderate | Maybe | Add deletion, expiration, DMCA, no reuse. | | Link to official California past exams | Low | Yes | Must avoid endorsement and replication. | | Host California exam questions | Moderate–High | Not initially | Needs counsel/permission. | | Host California selected answers | High | No | Selected answers add separate risk. | | Host real MEE questions | High | No | License needed. | | Store real-MEE-specific master packet bank | High | No | Looks like derivative answer-key layer. | | Use uploaded prompts to improve future rubrics | High | No | Converts user uploads into platform asset. | | Market “official MEE grading” | Very high | No | Trademark/copyright/advertising risk. | --- # Part F — Implementation Requirements ## F1. Must-Have for MVP User attestation before prompt upload. Async packet generation while student writes. Generic certified doctrine rubric catalog. Session-specific packet is transient by default. No backend storage of uploaded prompt. No backend storage of prompt-derived packet by default. No embeddings from user-uploaded prompts. No prompt text in logs or analytics. Optional browser-local packet save. Non-affiliation disclaimer. DMCA policy if any user material is stored. Marketing limited to “UBE-style,” “California-style,” “original simulated,” and “user-provided authorized materials.” --- ## F2. Should-Have for MVP Packet-generation confidence score. Fallback generic grading if packet is late. User-facing “grading framework ready” status. Admin redaction tools. Prompt deletion audit trail. Provider retention review. Automated log scanner for prompt leakage. Local packet export/delete controls. --- ## F3. Do Not Build Before Legal Clearance Real MEE question directory. Real MEE prompt matching. Real MEE packet catalog. NCBE-derived expected answer maps. NCBE point-sheet-derived scoring weights. State Bar selected-answer library. Cross-user reuse of user-uploaded prompt packets. Training/evals built from user-uploaded prompts. Marketing around “real MEE” or “official rubrics.” --- # Part G — Recommended Final Architecture ```text ┌──────────────────────────┐ │ SHEP Generic Rubrics │ │ Persistent / Certified │ └─────────────┬────────────┘ │ User uploads authorized prompt │ │ │ ▼ ▼ ┌──────────────────┐ ┌──────────────────────┐ │ Intake Session │────▶│ Subject/Issue Detect │ │ Attestation │ │ Fast Async Classifier │ └──────────────────┘ └──────────┬───────────┘ │ ▼ ┌──────────────────────┐ │ Session Packet │ │ Fact hooks + calls │ │ Transient/local only │ └──────────┬───────────┘ │ Student writes answer │ │ │ ▼ ▼ ┌──────────────────┐ ┌──────────────────────┐ │ Submit Answer │─────▶│ Grade Answer │ └──────────────────┘ │ Generic rubric + │ │ session packet │ └──────────┬───────────┘ │ ▼ ┌──────────────────────┐ │ Store Limited Result │ │ Score, issue tags, │ │ feedback, analytics │ └──────────────────────┘ ``` --- # Part H — Acceptance Criteria The feature is ready to ship only if all of the following are true: The upload page requires authorization attestation. The backend does not persist uploaded prompt text by default. The backend does not persist prompt-derived packets by default. Any saved packet is either local-browser-only or explicitly user-private, deletable, non-reused, and TTL-limited. No prompt text appears in logs, analytics, traces, queues, or error reports. No user-uploaded prompt is used for training, evals, or future rubric improvement. The generic rubric catalog contains only SHEP-authored doctrine rubrics. Marketing copy does not say “official,” “real MEE,” “NCBE-calibrated,” “past MEE bank,” or “official scoring.” California materials are linked, not replicated, unless counsel approves. A DMCA policy exists before backend storage of user-uploaded or user-derived materials. Provider retention settings match the privacy promise. --- # Bottom Line The best product architecture is: **SHEP-owned original prompts + reusable generic doctrine rubrics + runtime user-specific packet generation + no backend storage of prompt-derived packets by default.** The best marketing posture is: **Original simulated bar essay practice, user-provided authorized prompt support, and structured AI feedback.** The line to avoid is: **A platform-owned bank of real MEE or California-question-specific answer packets.** [1]: https://thebarexaminer.ncbex.org/terms-of-use/ "NCBE Bar Examiner Website Terms of Use - National Conference of Bar Examiners" [2]: https://www.ncbex.org/exams/mee/preparing-mee "MEE Preparation | NCBE" [3]: https://www.calbar.ca.gov/basic-page/past-exams "Past Exams | The State Bar of California" [4]: https://www.calbar.ca.gov/policies "Notices | The State Bar of California" [5]: https://www.calbar.ca.gov/state-bar-policy-web-links "State Bar Policy on Web Links | The State Bar of California" [6]: https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API?utm_source=chatgpt.com "IndexedDB API - MDN Web Docs - Mozilla" [7]: https://developers.openai.com/api/docs/guides/your-data "Data controls in the OpenAI platform" [8]: https://platform.claude.com/docs/en/build-with-claude/api-and-data-retention "API and data retention - Claude API Docs" [9]: https://www.law.cornell.edu/uscode/text/17/102 "17 U.S. Code § 102 - Subject matter of copyright: In general | U.S. Code | US Law | LII / Legal Information Institute" [10]: https://www.law.cornell.edu/uscode/text/17/106 "17 U.S. Code § 106 - Exclusive rights in copyrighted works | U.S. Code | US Law | LII / Legal Information Institute" [11]: https://www.ncbex.org/study-aids "Studying for the Bar | Bar Exam Practice | NCBE" [12]: https://www.law.cornell.edu/uscode/text/17/512 "17 U.S. Code § 512 - Limitations on liability relating to material online | U.S. Code | US Law | LII / Legal Information Institute" [13]: https://www.ftc.gov/business-guidance/resources/advertising-faqs-guide-small-business "Advertising FAQ's: A Guide for Small Business | Federal Trade Commission" [14]: https://www.ftc.gov/legal-library/browse/ftc-policy-statement-regarding-advertising-substantiation "FTC Policy Statement Regarding Advertising Substantiation | Federal Trade Commission"