Introduction to CID222

CID222 is an HTTP gateway your applications call instead of calling an LLM provider directly. It holds the provider credential, inspects the prompt before the provider sees it, inspects the reply before your application sees it, and records both.

The problem

An application that talks to a provider API sends whatever the user typed. That text carries customer names, card numbers, internal hostnames and occasionally a leaked API key, and once it reaches the provider it is out of your control. Nothing in the application stack reads the prompt, so nothing can mask it, refuse it, or record that it left the building. Nothing reads the reply either, so a model that echoes back the data it was given does so unobserved.

How CID222 does it

Your application posts to CID222 with a CID222 credential. CID222 resolves the provider credential for the calling tenant, runs the detection pipeline over the prompt, forwards a sanitised request, buffers the provider's reply, runs the pipeline again over the whole reply, and returns the filtered text.

The request path

1. Your application posts to POST /chat/completions with a gateway API key or a user JWT.
2. CID222 detects the language, then runs personally identifiable information (PII) detection, toxicity detection and jailbreak/injection detection in parallel over the prompt.
3. The decision maker resolves the detections into one action, in the priority order REJECT, then MASK, then FLAG.
4. On MASK or FLAG, the sanitised prompt goes to the provider under the tenant's stored credential. On REJECT the provider is never called, so no provider tokens are spent.
5. The provider streams token deltas back. CID222 accumulates them and forwards none of them.
6. When the reply is complete, CID222 runs PII and toxicity detection over the whole text, then emits the filtered reply as one Server-Sent Events (SSE) event followed by data: [DONE].

The detection fast path is approximately 150 ms. Total latency is dominated by step 5, because the reply is buffered: your client receives no text at all until the model has finished generating.

What CID222 detects

• PII and PHI. The shipped filter set is 11 filters naming 26 entity types — people, email addresses, phone numbers, credit cards, IBANs, national identifiers for several jurisdictions, passports, medical licences — matched by regular expressions and by an ONNX named-entity model.
• Leaked secrets. A dedicated filter group matches API keys and similar credentials in prompt text.
• Toxicity. A 13-label ONNX classifier scores the text. The default block threshold is 0.5, with per-label overrides shipped in the toxicity service's own threshold file.
• Prompt injection and jailbreak. An ONNX multi-label classifier plus a SQL-injection and cross-site-scripting scan. The block thresholds are baked into the model image at 0.85 and are not overridable by environment variable.
• Images and documents. Attached images are read by optical character recognition (OCR) and attached PDF, DOCX, TXT, CSV and XLSX documents are parsed, then both are inspected and redacted like text.
• Hallucination, in the background. When you send contexts, a retrieval-grounding check runs after the reply has already been returned. It writes a detection record; it never appears on the stream.

Every detection is written to the detections store with its entity type, confidence, filter name and the action taken, and is visible under Detection & Filtering → All Detections.

Multi-tenancy

Each tenant has its own provider credentials, filters and detection history. Credential resolution falls back from the tenant to the tenant group the tenant belongs to. Provider keys are stored as AES-256-GCM ciphertext, and gateway API keys are stored only as a SHA-256 hash — a key is shown once, at creation.

Providers and models

The seeded catalogue is:

GET /models returns only the models whose provider has an active credential for your tenant, so your list is usually shorter than this table.

Screenshot pendingDiagram: application to CID222, detection pipeline over the prompt, provider call, buffered reply, detection pipeline over the reply, single SSE event back to the application/assets/screenshots/gateway-request-path@0.4.png

Limits and known gaps

• The response stream is buffered, not incremental. The providers stream token deltas, but CID222 consumes them so that output filtering can judge the complete text. A client sees control events, then silence for the whole generation, then one filtered-response event carrying the entire reply. Interfaces built on the assumption of per-token deltas show a long pause and then a wall of text.
• There is no non-streaming mode. POST /chat/completions always returns text/event-stream. The request body accepts a stream field, but nothing reads it — setting stream: false changes nothing. A client that calls the JSON parser on the body fails.
• Unknown request fields are silently dropped. Global validation strips any body field the DTO does not declare, so a misspelled parameter is ignored rather than rejected, and the request succeeds with the wrong behaviour.
• Path prefixes are inconsistent. There is no global route prefix. Every surface is at its own root — /chat, /sessions, /models, /auth — except detection, which is the single surface under /api/v1.
• Gateway API keys do not reach every surface. /sessions, /image-analysis and /document-analysis accept a user JWT only. See Authentication.
• Only sign-in is rate limited. Login and the password-reset routes have per-IP limits. /chat/completions, /api/v1/guardrails/detect, /sessions, /models, /image-analysis and /document-analysis have no rate limiting at all, and the API returns no X-RateLimit-* headers.
• A tenant-group API key borrows a member's identity. The key is attributed to the earliest-added member of the group. Removing that user transfers the key's identity to another member rather than revoking the key.
• Detection quality is configuration-dependent. Accuracy and latency vary with text length, which detectors are enabled and how the tenant's filters are configured. Treat the 150 ms fast path as a shape, not a service level.

Related tasks

• Quickstart — issue a key, add a provider credential and make a first request that reads the stream correctly.
• Authentication — the two credential types and which surfaces accept each.
• Chat API — the full SSE event contract, event by event.