Architecture and services

CID222 sits between your applications and the model providers. It holds the provider credential, inspects what passes in both directions, and records the verdict — without asking the calling application to change how it talks to a model beyond changing the base URL.

The problem

A model provider sees whatever an application sends it. Once a prompt leaves the network there is no recall, no redaction and no audit trail: the customer record, the private key or the internal architecture in that prompt is now in someone else's logs. Blocking the provider outright stops the work; allowing it outright stops the governance. Something has to read the traffic, decide, and leave a record — in the request path, fast enough that nobody routes around it.

How CID222 does it

Components

• API gateway. A NestJS application on port 3000, bound to 0.0.0.0, with no global route prefix. Controllers declare their own base paths, so /chat/completions and /models sit at the root while the detection endpoint sits at /api/v1/guardrails/detect. It authenticates the caller, resolves the tenant, and orchestrates everything below.
• Detection pipeline. Five detectors that run in parallel over the same normalised text, plus a validation layer and a decision maker that turns their output into one action.
• Provider factory. One ChatProvider interface with an implementation per provider, so the orchestrator does not know which vendor it is talking to. Credentials are resolved per tenant at call time.
• Session manager. Conversation context and token accounting, held in PostgreSQL as JSONB.
• Audit trail. Every detection, every action and every administrative change lands in the database and is queryable through the admin API and the dashboard.

The services

Detection does not run in the gateway process. Each model is its own container, called over HTTP, so a model can be swapped or scaled without touching the gateway. The stack groups into seven jobs:

Published ports, compose profiles, memory limits and health checks for every one of them are generated from the compose file in services, ports and profiles.

Request flow

  client
    │  POST /chat/completions
    ▼
┌──────────────────────────────────────────────────────────────────────┐
│ nestjs-core                                                          │
│                                                                      │
│  1 authenticate ──► 2 normalise ──► 3 detect (in parallel)           │
│    JWT or key        decode            ├─ PII    → ml-detector       │
│    tenant + role     evasion           ├─ toxic  → hap-guard-v2      │
│                                        ├─ attack → attack-guard      │
│                                        └─ code   → auditor           │
│                                              │                       │
│                                              ▼                       │
│                                     4 validate + decide              │
│                                     reject > mask > flag             │
│                                              │                       │
│              ┌── reject ──► 400 to the client│                       │
│              │                               ▼                       │
│              └───────────────────► 5 provider call ─────────────────────►  OpenAI /
│                                              │                       │     Anthropic
│                                              ▼                       │◄─────────
│                                     6 filter the reply               │
│                                       PII + toxicity                 │
│                                              │                       │
│                                     7 log ──► postgres               │
└──────────────────────────────────────────────┼───────────────────────┘
                                               ▼
                                     one content event, then [DONE]

  background, off the request path:
    hallucination-guard   groundedness of replies that carried context
    risk-analyst          one review per user when the detection window fills

A chat request passes through these stages:

1. Authentication. A JWT or a gateway API key identifies the tenant. The role is re-read from the database on every request, so a token minted before a demotion does not keep the old role.
2. Normalisation. Evasion decoders run first — ROT13, base64, leetspeak, Unicode homoglyphs, case — so a detector sees the decoded text rather than the disguised one.
3. Parallel detection. PII, toxicity, jailbreak and injection, and code safety all run at once over the normalised text.
4. Validation and decision. Checksum and context rules confirm or demote each detection, and the decision maker folds them into one action. The strongest action wins: reject beats mask, mask beats flag.
5. Provider call. The surviving text, masked where masking applied, goes to the provider on the tenant's own credential.
6. Response filtering. The reply is scanned for PII and toxicity in real time before any of it reaches the client.
7. Logging. Detections, token usage and cost are written against the session.

What the client sees

The chat surface is always text/event-stream; there is no non-streaming mode. The stream is CID222's own contract, not the provider's. The orchestrator consumes the provider's token deltas, accumulates the whole reply, runs output filtering over it, and emits the result as one content event. A client sees optional control events, then a pause while the model generates, then a single large content event, then [DONE].

Storage and caching

PostgreSQL holds tenants, credentials, filter rules, sessions, detections and the audit trail. Session context and token usage are JSONB columns on the session row. Redis backs the queues, the distributed locks and the sliding-window rate limiter.

Background work

Queues carry the work that must not sit in the request path: asynchronous document redaction, SIEM and webhook export, repository tracking, response-risk inspection, and the LLM review pipeline. The review pipeline is not a per-message job — repeated jailbreak and injection detections fill a sliding window per user, and crossing it opens one review by the local risk-analyst service.

Deployment

CID222 is deployed on infrastructure you control. It ships as a Docker Compose stack, and as an appliance image (OVA and ISO) that carries every model in the image so it can be installed with no outbound connectivity at all. There is no hosted control plane: an air-gapped appliance is fully functional, including licence verification.

Limits and known gaps

• A reverse proxy is not a forward proxy. Putting CID222 in front of your application governs what that application sends. It does not intercept a browser going directly to a provider's website. That needs the inline-proxy or endpoint deployment, which decrypt the session with their own CA.
• Almost nothing is rate limited. The sliding-window limiter guards login, password reset, the help assistant, browser-extension sign-in and attestation, the repository webhook, and a user's own unlock request. /chat/completions, /api/v1/guardrails/detect, /sessions/*, /image-analysis/*, /document-analysis/* and /models have no limit of any kind. A tenant quota is not an implemented control; budget for it at the load balancer.
• The stream is not incremental. Because output filtering runs on the complete reply, the client waits for the whole generation and then receives it at once. A user interface that expects token-by-token rendering will look frozen and then jump. This is the cost of filtering the response before the user reads it.
• Chunked and resumable uploads are refused, not inspected. No path buffers a body across requests, so a fragmented upload cannot be reassembled and scanned. All paths refuse them by default rather than passing uninspected bytes.
• Unknown request fields are dropped silently. The global validation pipe strips anything the DTO does not declare instead of rejecting it, so a misspelled field produces a successful request that ignored your intent.
• The error envelope is not uniform. Structured errors carry {statusCode, code, message}, but plain framework exceptions return {statusCode, message, error} with no code. Do not write a client that assumes code is always present.
• Retention is configuration, not a product default. How long clean traffic is kept is set at runtime per deployment. Redacted and blocked events are retained as policy evidence regardless.

Related

• Multi-tenancy — how one deployment serves several tenants.
• The content safety pipeline — what runs inside stage 3.
• Chat API — the stream contract in full.