# BYOP Technical Feasibility Analysis
*Updated after reviewing Vercel AI Gateway docs — this supersedes the prior version on several points*

---

## The Big Revision: ZDR Changes the Gateway Analysis

My earlier analysis said the Vercel AI Gateway was a poor fit for the BYOP inference path because its logging features would capture prompt data. **That was wrong.** Vercel has a documented Zero Data Retention (ZDR) feature with actual contractual agreements in place — both Vercel's own infrastructure and the downstream AI providers.

When ZDR is enabled:
- **Vercel's gateway** permanently deletes prompts and responses immediately after the request completes. This is the gateway's *default* behavior for its own infrastructure.
- **The provider** (Anthropic, Google Vertex, Azure, Groq, Amazon Bedrock, and ~9 others) is also contractually bound to not retain the prompt — Vercel has negotiated these agreements directly.
- Per-request enforcement is `providerOptions: { gateway: { zeroDataRetention: true } }`. Team-wide enforcement is a dashboard toggle.
- ZDR is a superset of "disallow prompt training" — both are covered.

This is exactly the "provider confidentiality requirements" the memo says you should be seeking from your vendor. Vercel has already brokered those contracts. It costs $0.10 per 1,000 requests for team-wide, nothing extra for per-request.

**The memo's framing of the gateway as a logging risk was correct for the default configuration, but incorrect once ZDR is applied.** With ZDR on, the gateway becomes a viable part of the architecture.

---

## The Recommended Architecture: Browser → Edge Function → Gateway (ZDR) → Provider

The cleanest path is a hybrid that achieves "your main servers never touch the prompt" while keeping you in control of auth:

```
[Browser]
  ↓  (1) User types prompt, clicks "grade"
  ↓  (2) Browser requests a short-lived session token from your SvelteKit backend
[SvelteKit Backend]
  ↓  (3) Issues a signed JWT (your code, your signing key) — 5-minute TTL, single session
  ↓  (4) JWT returned to browser
[Browser]
  ↓  (5) Browser sends prompt + JWT to a thin Vercel Edge Function
[Vercel Edge Function — your code, no logging]
  ↓  (6) Validates JWT, injects AI Gateway API key, adds zeroDataRetention: true
  ↓  (7) Forwards to AI Gateway
[Vercel AI Gateway — ZDR enforced]
  ↓  (8) Routes to ZDR-compliant provider (Anthropic, etc.)
[AI Provider — ZDR contract in place]
  ↓  (9) Inference result flows back to browser
[Browser]
  ↓  (10) Browser strips/validates the response (n-gram check — see below)
  ↓  (11) Sends only the scoring artifact (numbers + generic labels) to SvelteKit backend
[SvelteKit Backend]
  → Stores the scoring artifact (never sees the prompt)
```

**What this achieves:**

| Claim | Achievable? | How |
|-------|-------------|-----|
| "We don't receive the prompt" | ✓ | SvelteKit backend only sees JWT issuance + final scores |
| "We don't retain the prompt" | ✓ | Main backend never has it; Edge Function has no logging |
| "The provider doesn't retain the prompt" | ✓ | ZDR contract via Vercel agreement |
| "Vercel doesn't retain the prompt" | ✓ | Vercel's own ZDR policy (default behavior, plus contractual at provider level) |
| "We don't store prompt-specific derivatives" | ✓ | Output minimization (separate design change, see below) |

This is a strong factual posture. Not perfect — the prompt exists in plaintext in the Edge Function for the duration of the request — but the Edge Function is your code running on Vercel infra, and with ZDR applied, neither your code nor Vercel's layer retains it.

---

## Credential Scoping Gap (and the Workaround)

**The gap:** The AI Gateway does not natively support issuing per-user, expiring, scoped browser credentials. It has two auth methods:
- Permanent API keys (team-scoped, don't expire unless revoked)
- OIDC tokens (12-hour, auto-issued by Vercel for your project's server environment — not for browsers)

You cannot hand a permanent API key to the browser. That would let any user make arbitrary calls to the gateway on your bill.

**The workaround** is the JWT flow described in step 2-6 above:
- Your SvelteKit backend is the only place the real AI Gateway API key lives
- Your backend issues a short-lived signed JWT (5-minute TTL, bound to the user's session ID)
- The Edge Function validates the JWT signature + expiry before injecting the real key
- The Edge Function explicitly has no request body logging

This isn't "no credential on the backend" — you still have the gateway key on the server — but the key difference is that **the prompt never arrives at the server.** The backend touches only the JWT handshake. That's the factual claim that matters.

---

## Output / Storage Minimization

**No change from the earlier analysis — still highly feasible.**

Stop persisting prompt-specific packets. The session packet (rule atoms, issue archetypes keyed to the specific prompt) becomes ephemeral — it lives only in the browser during the practice session and in the inference response. What gets stored in the database: subject classification, rubric dimension scores (numbers), and generic feedback phrased without reference to the prompt's fact pattern.

This is a moderate refactor of the three-layer packet engine. The generic rubric layer (which you author, keyed to subject + call type) stays. The transient session packet stays transient — don't persist it. The persisted results layer stores only scores + generic labels.

---

## Anti-Regurgitation: n-gram in the Browser

**Verdict: Not hard. Run it client-side.**

In the architecture above, the browser has both things you need to compare: the prompt (typed by the user) and the AI's response (returned from the Edge Function). You don't need the server involved at all for this check.

**Implementation:**

```typescript
// Word 5-gram Jaccard similarity — runs in the browser, ~10ms for typical texts
function ngramJaccard(a: string, b: string, n = 5): number {
  const ngrams = (text: string) => {
    const words = text.toLowerCase().split(/\s+/);
    const set = new Set<string>();
    for (let i = 0; i <= words.length - n; i++) {
      set.add(words.slice(i, i + n).join(' '));
    }
    return set;
  };
  const sa = ngrams(a);
  const sb = ngrams(b);
  const intersection = [...sa].filter(g => sb.has(g)).length;
  const union = new Set([...sa, ...sb]).size;
  return union === 0 ? 0 : intersection / union;
}

// Before submitting scores to backend:
const similarity = ngramJaccard(userPrompt, aiResponse);
if (similarity > 0.15) {
  // Response quotes too much of the prompt — don't submit this artifact
  // Show user a warning or regenerate in "safe mode"
}
```

The 0.15 threshold (15% n-gram overlap) is a starting point — you'd tune it. Normal grading feedback has very low overlap with the fact pattern. Verbatim regurgitation would score 0.4+.

**Why client-side is fine here:** The legal goal isn't to make this check tamper-proof — a determined user can always bypass client-side JavaScript. The goal is to (a) prevent the system from *accidentally* storing prompt-derived content, and (b) demonstrate that the platform has enforcement in place. Client-side n-gram satisfies both: it's a real control that catches accidental regurgitation, and its presence is evidence of intent. If a user deliberately bypasses it, liability shifts to them under the terms.

The n-gram check is also useful on the server side — not against the prompt (you don't have it) but as a lint check against stored artifacts. You can check stored artifacts for patterns that look like exam fact patterns: high proper noun density, numbered calls ("discuss whether X"), citation formats. Not a corpus match — just a structural red-flag detector.

---

## One Caching Warning

The AI Gateway's automatic response caching feature would be dangerous for the BYOP path. If the gateway caches a response keyed to the prompt, it could serve that cached response to other users who submit the same prompt. This would:
- Make the gateway a de facto prompt/response repository
- Violate ZDR even if the original request was ZDR-compliant

**Disable caching explicitly on the BYOP inference path.** The docs note that caching happens at the provider level and ZDR-compliance of caching depends on the provider — don't assume ZDR disables caching automatically.

---

## What to Drop (BYOK and Confidential Compute)

**BYOK:** You already ruled this out. Confirmed correct — adds user friction, doesn't materially improve the legal posture beyond the architecture above, and complicates billing.

**Confidential computing:** No mainstream AI provider offers this for general API use. Not buildable now.

**"Unreadable to provider" cryptography:** As the memo correctly states — not achievable with hosted LLMs. FHE for transformer inference is computationally infeasible. The ZDR contractual approach is the practical substitute.

---

## Recommended Build Order

1. **Output minimization** — stop persisting session packets. Purely server-side. Directly kills the "derivative answer key" risk. Start here.
2. **Browser → Edge Function → Gateway (ZDR) flow** — this is the core architecture change. The Edge Function is a small piece of code (under 50 lines). The credential JWT flow is straightforward.
3. **Client-side n-gram check** — add before artifact submission. Small utility function, runs in the browser.
4. **Structural red-flag detector on stored artifacts** — server-side lint on what gets persisted. No corpus needed.
5. **BYOP gating / risk scoring** — structural prompt analysis (length, question format, citation patterns) to flag likely NCBE content and trigger a "safe mode" that doesn't persist anything beyond the raw score.

---

*Sources: [Vercel ZDR docs](https://vercel.com/docs/ai-gateway/capabilities/zdr) · [Vercel AI Gateway Capabilities](https://vercel.com/docs/ai-gateway/capabilities) · [Vercel Gateway Auth](https://vercel.com/docs/ai-gateway/authentication-and-byok/authentication)*