Inline proxy and ICAP problems
Callers the inspection endpoint refuses, source addresses that are not what they look like, and a proxy CA that will not download.
- Version: 0.4
- Role: admin_user
- Type: troubleshooting
Inspection fails in two places: admission (may this caller talk to the listener?) and content (can this request be inspected at all?). Both produce refusals that read like faults.
Note
The ICAP listener cannot authenticate its peer. No SWG product CID integrates with offers a credential or custom-header field, so admission is by address alone, and an admitted peer is believed down to the user it claims. Treat the source-IP allowlist as the only control there — and read the next section before trusting it.
Find your symptom
| What you see | Runbook |
|---|---|
| The SWG gets 401 or 403 from the inspection endpoint | The inspection endpoint refuses the caller |
| Every caller arrives as the same docker bridge address | Source addresses are not what they look like |
| An allowlist admits every caller, or none | Source addresses are not what they look like |
| The CA download fails, or clients warn on every intercepted site | The proxy CA is not available |
| A large or chunked upload is refused as not inspected | Chunked uploads are refused, not inspected |
Admission
The inspection endpoint refuses the caller
Admission is evaluated in a fixed order: an mTLS requirement first, then a bearer API key, then network position. A key that resolves to no tenant is refused outright rather than treated as no key — otherwise a key holder could attribute records to any tenant they named.
Note
allowed_source_ips and mtls_subject_dn were written, displayed and read by nothing until
2026-08-29. Because a value set before then was never applied, the upgrade clears both and
records the previous values in the event log — a first unattended application could otherwise
have 403'd every caller. Re-enter them deliberately.
The inspection endpoint refuses the proxy or gateway calling it
You might see: Client certificate required · API key required · Invalid API key · Internal API only
Before you start
Checks
- 1
Read whether this deployment mandates a client certificate
expected · Either no DN is required, or the caller presents a certificate matching it.
- 2
Read whether the caller presents a gateway API key
expected · The caller sends `Bearer cid_key_…` and the key resolves.
- 3
Check whether the call arrives through a reverse proxy
expected · Either the caller presents a key, or it reaches the listener directly from an internal address.
- 4
Before trusting any source-IP control, check what address the caller actually arrives as
expected · Callers arrive with their own addresses.
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | mtls-required-not-presented (critical) | An mTLS requirement is an additional constraint, evaluated first. It was written, displayed and read by nothing until 2026-08-29; because a value set before then was never applied, the upgrade clears it and records the previous value in the event log. A value re-entered after the upgrade does enforce. Fix: Present a client certificate whose subject DN matches the configured value, or clear the DN if this deployment does not run mTLS. Re-read the event log for the value the upgrade cleared before re-entering one. Verify: expected · It is accepted. |
| check 2 fails | key-missing-or-unresolvable (critical) | Returning "no tenant" for a key that resolves to nothing would be indistinguishable from a call with no key, and callers fall back to the tenant named in the body in that case — which would let a key holder attribute records to any tenant. So the key is refused instead. Fix: Issue a gateway API key for the caller and configure it as a bearer token. Follow A gateway API key is refused when the key exists and is still refused. Verify: expected · It is accepted and the resulting records carry the expected tenant. |
| check 3 fails | keyless-through-edge (warning) | Behind a reverse proxy every caller looks internal, so the private-network allowance would admit the whole internet. The listener refuses keyless calls that came through an edge and says so. Fix: Give the caller a gateway API key, or have it reach the listener directly. Verify: expected · It is accepted. |
| check 4 fails | source-ip-is-the-bridge (warning) | A SWG running as a VM beside CID222 arrives at the listener as the docker bridge address, because the source network's own NAT masquerades it. A source-IP allowlist in that situation admits every caller or none, and proves nothing either way. On a physical LAN the address survives and the allowlist works. Fix: Follow Every ICAP caller arrives as the same docker bridge address. Verify: expected · The caller arrives with its own address. If that does not apply: Attach the support bundle and the listener's rejection log lines — they name the apparent source address and the reason. |
Source addresses are not what they look like
Lab-measured: two SWG virtual machines beside CID222 both arrived at the listener as the same docker bridge address. The cause is the source network's own NAT, not CID and not Docker's userland proxy. On a physical LAN the address survives and the allowlist works — this is a property of same-host VM labs.
The listener detects the condition and logs it once. Read that log before trusting any source-IP control.
Every ICAP caller arrives as the same docker bridge address
You might see: every SWG appears as 172.19.0.1 · the ICAP source-IP allowlist does not distinguish our proxies · allowed_source_ips has no effect · the allowlist blocked every caller after we set it
Before you start
Checks
- 1
Read how the ICAP listener admits callers
diagnostics · posture.inspectionSourceIpMode = "unrestricted"
- 2
Check whether callers arrive with their own addresses or with the container bridge address
expected · The recorded source address is the gateway's own address.
- 3
When an explicit allowlist is set, verify it lists every caller that must be admitted
expected · Every caller is listed.
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 2 fails | nat-rewrites-the-source (warning) | A gateway running as a virtual machine beside CID222 does not reach the listener with its own address. The VM network's own masquerade rule rewrites the source once the destination leaves that subnet, so both a multipass and a libvirt guest were measured arriving as the same container bridge address. This is a property of same-host laboratories: a gateway on a physical LAN keeps its address and the control works. Fix: Do not rely on a source-IP allowlist in a same-host VM laboratory — it cannot distinguish the callers, and setting it either admits all of them or none. To test an allowlist in that environment, add a rule that returns the container subnet's traffic ahead of the masquerade rule. In production, place the gateway on the network rather than on the appliance host. Verify: expected · The address is the gateway's own, not the container bridge address. |
| check 1 fails | unrestricted-admission (warning) | The listener cannot authenticate its peer — no product it integrates with offers a credential or a custom header — so admission is by address alone, and an admitted peer is believed down to the user identity it claims in each request. Unrestricted admission means anything that can reach the port can assert any user. Fix: Restrict the listener to the addresses of the gateways that legitimately call it, and place the port on a segment those gateways alone can reach. Treat network placement, not the allowlist, as the real control. Verify: diagnostics · posture.inspectionSourceIpMode = "unrestricted" |
| check 3 fails | allowlist-replaced-the-default (critical) | An explicit allowlist replaces the private-range default rather than extending it. Setting one address therefore removes admission for every other caller, including ones that worked a moment earlier, and the failure appears at the gateway as a refusal rather than at the listener as a configuration change. Fix: List every caller that must be admitted, not only the one you were adding, then re-test each gateway. Values written before this control began to be enforced were cleared during the upgrade and recorded in the event log, so a value you remember setting may no longer be in place. Verify: expected · Every one is admitted. |
The proxy itself
The proxy CA is not available
The forward proxy is opt-in and its CA lives in its own state. Recreating the container with a fresh volume generates a new CA, and every client that trusted the old one must be given the new one — old trust does not transfer.
Content
Chunked uploads are refused, not inspected
No path buffers across requests, so a chunked or resumable upload cannot be inspected. Since
2026-08-29 all three paths refuse them by default (CID_RESUMABLE_UPLOAD_POLICY on ICAP and
the forward proxy; the extension honours its fail-closed setting and records a visible "not
inspected" event) rather than passing them through.
Reassembly — buffer by upload id, inspect at finalize, re-fragment — is not implemented, so a user sending a large file gets a refusal rather than a redaction. That is the fail-closed choice working, not a fault.
A file is refused, or its analysis fails
You might see: No file uploaded (expected multipart field "file") · Image size exceeds maximum allowed size of 10MB · Unsupported file type. Upload a PDF, DOCX or TXT. · legacy and macro-enabled spreadsheet formats cannot be safely redacted
Before you start
Checks
- 1
Read whether the format is one the product accepts at all
expected · The format is accepted on that surface.
- 2
Read the size limit for that surface
expected · The file is under the limit named in the message.
- 3
Check whether the client was doing a chunked or resumable upload
diagnostics · posture.resumableUploadPolicy = "block"
- 4
Read whether document analysis is switched on for this deployment
expected · The service is enabled.
- 5
Read the document parser's own health
diagnostics · services[document-parser].healthy = true
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | format-not-accepted (info) | Some formats are refused on purpose rather than partially handled. A macro-enabled spreadsheet cannot be redacted safely, and a redaction that silently missed content would be worse than a refusal. Fix: Save the file in an accepted format — a spreadsheet as .xlsx without macros, a document as PDF, DOCX or TXT — and upload it again.Verify: expected · It is accepted. |
| check 2 fails | over-size-limit (info) | Each surface caps what it will buffer, because analysis holds the whole file in memory and an unbounded upload is an out-of-memory kill for whichever detection service is largest. Fix: Split or compress the file, or use the document path rather than the image path for large scans. Verify: expected · It is accepted. |
| no check points here | resumable-refused-by-policy (warning) | No path buffers across requests, so a chunked or resumable upload cannot be inspected. Rather than passing it through uninspected, all three paths refuse it by default and the extension records a visible "not inspected" event. Reassembly — buffer by upload id, inspect at finalize, re-fragment — is not implemented. Fix: Have the user upload the file in a single request, or accept the risk explicitly by changing the policy. Do not read a refusal here as a product fault: it is the fail-closed choice working. Verify: diagnostics · posture.resumableUploadPolicy = "block" |
| check 4 fails | parser-disabled (warning) | Document analysis is a separate service and can be switched off, for instance on a minimal development stack. The endpoint answers 503 with a distinct message so it is not confused with a service that is enabled and down. Fix: Enable the document parser and restart the gateway so it re-reads the setting. See File redaction. Verify: expected · The analysis runs. |
| check 5 fails | parser-unhealthy (critical) | The gateway holds a circuit breaker in front of the parser: once it has failed repeatedly the gateway stops calling it and answers "temporarily unavailable" immediately, which is why the failure can look instant. Fix: Follow One service is unreachable or reports itself unhealthy. Verify: diagnostics · services[document-parser].healthy = true If that does not apply: Attach the support bundle and the file's type and size. Do not attach the document itself unless policy allows it to leave the estate. |
Related
- Inline proxy — deploying the forward proxy.
- ICAP integrations — per-vendor SWG configuration.
- Inspection — the policy these controls come from.
- Access keys — issuing the bearer key a caller presents.
Last updated on