Install and activation problems
The failures that happen between booting the installer and a licensed appliance serving traffic, with the runbook for each.
- Version: 0.4
- Role: admin_user
- Type: troubleshooting
Everything between booting the installer and a licensed appliance serving traffic. Find the symptom in the table, then work the runbook under it.
Nothing about the product is collected until the product runs, so anything that fails during provisioning is diagnosed from the installer's own console messages and its log at /var/log/cid-firstboot.log. From first boot onwards, start with the diagnostic snapshot instead.
Find your symptom
| What you see | Phase | Runbook |
|---|---|---|
| The run stops with a red failure line naming the registry, at step 6/8 | Install | Provisioning stops at 6/8 |
| The machine has no address, or only a link-local one | Install | No DHCP on this network |
| The installer stops before downloading anything, quoting memory or free disk | Install | The resource check fails |
| Name resolution works and nothing gets out | Install | The appliance cannot get out |
| The install finished and the browser shows a blank page or refuses the connection | First boot | The dashboard does not load |
| The wizard's connectivity step returns a server error | First boot | The connectivity step returns 500 |
| The wizard completes and the appliance asks for setup again | First boot | The wizard loops |
| Settings you save disappear after a restart | First boot | Saved state does not stick |
| The wizard's provider step shows an empty list | First boot | The providers step is empty |
| A licence upload is rejected as a bad signature | Licensing | License upload fails with 400 |
| The appliance can only ever run the trial | Licensing | No licence trust anchor |
| Every product route answers 402 as soon as setup finishes | Licensing | 402 right after setup |
| The licence was working and now the product is locked | Licensing | The licence has expired |
| Nothing answers on 443 | TLS | Nothing is listening on 443 |
| The browser reports an expired certificate | TLS | The certificate is expiring |
| The update page shows an obviously wrong version | Updates | The version is unknown |
| The update check finds nothing, or fails | Updates | The channel is unreachable |
| Dates are wrong: a valid licence looks expired, a valid certificate looks invalid | Any | The clock is not synchronised |
Install
Provisioning stops at 6/8 container images
The install is not broken: the machine has no working path to the registry. Read the indented line above the failure — it is the pull tool's own message, and it is what separates a blocked port from a broken resolver, a stale credential and an inspected TLS connection.
install-provisioning-stops-at-images is not in this bundle. Run npm run docs:sync to pull the current corpus.This network has no DHCP
The default boot entry is unattended and expects an address to be offered. A segment that offers none needs the boot entry that asks for the address before touching the disk, or a static address set from the appliance console.
This network has no DHCP, and the appliance comes up with no address
You might see: no IPv4 network (no default route) · the VM summary shows only an fe80:: address · the appliance has no IP address after the install · this network has no DHCP
Before you start
Checks
- 1
Check which boot menu entry this appliance was installed from
expected · The install was started, or will be started, from the "set the network by hand — no DHCP" entry.
- 2
Check whether a static address was set from the appliance console instead
expected · An IPv4 address, gateway and name servers are set, and a default route exists.
- 3
Verify the name servers you configured actually answer
expected · Both queries answer. A configured resolver that times out is worse than none, because the failure looks like a network outage.
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | wrong-boot-entry (warning) | The default boot entry is fully unattended and expects DHCP; it never asks for an address. On a segment with no DHCP that entry produces a machine with no network and an install that stops at the first pull. The dedicated entry shows a network page before it touches the disk. Fix: Reinstall from the boot entry "set the network by hand — no DHCP". The installer shows its own network page: select the interface, choose Edit IPv4 → Manual, fill in the subnet, address, gateway and name servers, then select Done. Nothing else is asked, and the address you type is the address the finished appliance keeps. Verify: expected · The appliance holds the address you typed, and provisioning ran to completion. |
| check 2 fails | no-static-address (critical) | No address was configured by hand and none was offered by the network, so the appliance has only a link-local address. Every outbound step — the registry pull, name resolution, the update channel — fails identically, which makes the cause hard to read from the symptoms. Fix: Set the address from the appliance console rather than reinstalling: press Alt+F1 for the console, then F2, and set the address, gateway and name servers. The console uses a US keyboard layout and warns when Caps Lock is on. Reboot afterwards — provisioning resumes by itself and keeps everything already downloaded. Verify: expected · The address you set is present and a default route exists. |
| check 3 fails | resolver-unreachable (critical) | A static configuration includes the name servers, and a wrong entry there produces an appliance that has an address and still resolves nothing. The installer then reports that it cannot resolve the registry with this network's DNS, with public resolvers, or from a pinned address. Fix: Correct the name servers on the console network page and reboot. If the site's resolvers cannot answer for the registry at all, add the address your CID222 contact gives you to /etc/hosts as a temporary bypass for the install window. Verify: expected · The query answers with the expected address. |
The resource check fails
The installer measures memory and free disk before it downloads anything, because an undersized machine otherwise fails an hour later as containers that will not stay up. Its own thresholds predate the current supported minimum, so a run that only warned can still be undersized.
The installer stops on memory or free disk before anything downloads
You might see: this machine has 4096 MB of RAM — the appliance cannot run below 6 GB · only 12 GB free on / — the release images need about 40 GB · memory is 8192 MB; 16 GB is the designed size · the installer stopped at 0/8 this machine
Before you start
Checks
- 1
Read the memory the installer measured on this machine
expected · The reported memory is at least 32 GB, the supported minimum for the full stack.
- 2
Read the free disk the installer measured on the root filesystem
expected · At least 100 GB free, the supported minimum. The installer refuses below 20 GB and warns below 45 GB.
- 3
Check whether the kernel stopped the previous attempt
expected · No line reports "KILLED BY THE KERNEL".
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | ram-too-small (critical) | The installer checks memory and free disk before it downloads anything, because an undersized machine otherwise fails an hour later as containers that will not stay up. It refuses below 6 GB and warns below 16 GB — but 16 GB was the previous design value, and the supported minimum for the full stack is now 32 GB. A run that produced only a warning can therefore still be a machine that will be OOM-killed under load. Fix: Power the machine off, raise the memory to at least 32 GB for the starter and professional packages or 64 GB for enterprise, and boot again. Images already downloaded are kept, so the retry is quick. See Sizing and requirements. Verify: expected · The step reports the raised memory and the run continues past it. |
| check 2 fails | disk-too-small (critical) | The release image set needs about 40 GB to download and unpack, and an update needs room for a second copy of every image on top of that. The installer refuses below 20 GB free and warns below 45 GB; the supported minimum for a production appliance is 100 GB. Fix: Give the machine a larger disk and reinstall. Retrying on the same disk cannot succeed — the installer's own message says so. Growing the virtual disk without growing the filesystem does not help either; the check reads free space on the root filesystem. Verify: expected · At least 100 GB free on the root filesystem. |
| check 3 fails | oom-during-install (critical) | The kernel killed the previous attempt for running out of memory. The installer keeps the reason and reports it on the next boot, because a machine that silently restarts its install looks like a network fault. Unpacking the image set is the memory peak of the whole install. Fix: Raise the memory to at least 32 GB and boot again. Everything already downloaded is kept. If the machine is already at 32 GB and is still killed, collect the console output and the log before changing anything else. Verify: expected · The run completes without a kernel kill. If that does not apply: Attach /var/log/cid-firstboot.log and the hypervisor's own memory configuration for the VM. |
The appliance cannot get out
Name resolution, a TCP connection on 443, and an authenticated request through the site proxy fail independently. Each of them passes routinely while the install still fails, which is why "this network has full internet access" is usually said in good faith and is usually about a different path.
The appliance cannot get out — one or more egress tests failed
You might see: egress tests failed · the appliance cannot reach the registry · ETIMEDOUT after 5000ms · proxy returned 407 Proxy Authentication Required
Before you start
Checks
- 1
Verify the appliance actually ran its outbound probes
diagnostics · appliance.egress
- 2
Read the name-resolution results for the hosts this appliance must reach
diagnostics · appliance.dns
- 3
Read whether an outbound proxy is configured on the appliance
diagnostics · appliance.proxy.configured = true
- 4
Read whether the release channel answered on the last attempt
diagnostics · appliance.update.channelReachable = true
- 5
Check whether a TLS-inspection device is presenting its own certificate to the appliance
expected · The issuer is a public certificate authority, not your organisation's inspection CA.
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | probes-disabled (info) | The outbound probes are switched off for this deployment, so the snapshot carries no evidence either way and no egress finding can fire. This is a deliberate setting for a box that must make no outbound request at all. Fix: Re-enable the diagnostics egress probes on the gateway container and collect the snapshot again, or run the three tests by hand from the appliance host. The probes are the only outbound requests the appliance makes on its own. Verify: diagnostics · appliance.egress |
| check 2 fails | dns-broken (critical) | The appliance resolves with this network's resolver. A resolver that cannot answer for the provider or channel hosts fails every outbound call identically, and a browser on a laptop keeps working because it resolves over DNS-over-HTTPS. Nothing downstream of resolution can be diagnosed until this is fixed. Fix: Set a resolver that answers for the hosts in the DNS section of the snapshot, then collect the snapshot again. See Network prerequisites. Verify: diagnostics · appliance.dns |
| check 3 fails | proxy-not-configured (critical) | This site requires an outbound proxy and the appliance has none, so every outbound connection is attempted directly and times out. A timeout reads as a dead network rather than as a missing setting, which is why the network is reported as open. Fix: Configure the outbound proxy on the appliance and restart the gateway container so it re-reads the setting. Docker itself reads neither the shell environment nor apt's configuration, so a proxy set for the shell alone does not reach image pulls. Verify: diagnostics · appliance.proxy.configured = true |
| check 4 fails | https-path-blocked (critical) | Names resolve and the HTTPS path does not complete. DNS, a TCP connection on 443, and an authenticated HTTPS request fail independently: a firewall that permits ICMP and blocks 443 is ordinary, and Ubuntu's mirrors are served over port 80, so ping and apt both succeed while this fails. The three tests are not interchangeable. Fix: Allow direct outbound HTTPS from the appliance to the hosts the snapshot names as failing, or route them through the site proxy and configure it on the appliance. Read each failing target's detail line in the snapshot before changing a rule — a 407 is a proxy credential problem, a timeout is a blocked path, and a TLS error is inspection. Verify: diagnostics · appliance.update.channelReachable = true |
| check 5 fails | tls-inspection-untrusted (critical) | A middlebox terminates TLS and presents a certificate the appliance does not trust, so the connection fails at the handshake. The detail line reports a TLS error rather than a timeout, which is the only way to tell this apart from a blocked port. Fix: Install your site's CA certificate on the appliance so it trusts the inspection device, or exempt the appliance's outbound destinations from inspection. Both are legitimate; pick the one your security team already operates. Verify: expected · The failing egress targets report ok, or the issuer is now trusted by the appliance. |
First boot
The dashboard does not load
An appliance installed from the ISO serves HTTPS only, and the front end refuses to start with a certificate it cannot use rather than serving a broken one — so a blank page, a refused connection and a brief upstream error have three different causes and one page.
The install finished but the dashboard does not load
You might see: the browser shows a blank white page · https://<ip>/ refuses the connection · HTTPS sidecar FAILED · 3 of 16 containers are running; these are not:
Before you start
Checks
- 1
Read the console's own count of running containers
expected · Every container is running, and no line reports "N of M containers are running".
- 2
Verify something holds port 443 on the appliance host
diagnostics · appliance.tls.servedBy
- 3
Verify the gateway itself is reachable
diagnostics · services[nestjs-core].reachable = true
- 4
Verify the gateway container is not restarting repeatedly
diagnostics · containers[nestjs-core].restarts = 3
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 2 fails | https-listener-not-installed (critical) | The HTTPS front end is installed by a separate step at the end of the run, and it is not optional: the dashboard sends an upgrade-insecure-requests policy, so a plain http:// page renders blank and https:// is refused outright when the listener never installed. The console reports this at the end of the run. Fix: Re-run the HTTPS installer on the appliance host with sudo bash /opt/cid-install/install-https.sh, then reload the dashboard over https://. The appliance serves a self-signed certificate at this point, which is correct — it has no name and no CA yet, and the site replaces the certificate later.Verify: diagnostics · appliance.tls.servedBy |
| check 1 fails | containers-not-up (critical) | The install itself succeeded and one or more services did not stay up. The console names each one and prints the last three lines of its log, which is where the reason is. An exit code of 137 is a kernel out-of-memory kill and means the machine is undersized, not that the service is broken. Fix: Read the last lines the console printed for each container that is down. For exit 137, raise the machine's memory to at least 32 GB and reboot — see The machine is below the package's RAM minimum. For anything else, read the full log with docker compose -f /opt/cid/docker-compose.release.yml logs.Verify: expected · The summary reports the full container count running and names none as down. |
| check 3 fails | gateway-down (critical) | The dashboard is a static frontend that calls the gateway for everything. When the gateway does not answer, the page can load and then fail every request, or the front end can return 502 while it waits for an upstream that never arrives. Fix: Read the gateway container's log from the most recent start and fix the boot error it reports. A placeholder JWT secret refuses boot by design, and pending migrations stop the readers of the new columns — both surface here as a gateway that will not serve. Verify: diagnostics · services[nestjs-core].reachable = true If that does not apply: Attach the support bundle, which carries the gateway log tail. |
| check 4 fails | gateway-crashloop (critical) | The gateway is restarting, so requests that land mid-restart fail with errors unrelated to what the reader was doing. A sustained restart count is the signature of a boot failure, not of load. Fix: Follow A container keeps restarting, which separates an out-of-memory kill from a configuration refusal. Verify: diagnostics · containers[nestjs-core].restarts = 3 |
The connectivity step returns 500
Almost always a permissions problem on the appliance's state directory rather than a network problem. The connectivity probe itself may well have succeeded before the write failed.
The setup wizard's connectivity step returns 500
You might see: POST /setup/connectivity → 500 · setup wizard connectivity test fails with a server error · Internal server error on the network step of the wizard · wizard will not advance past connectivity
Before you start
Checks
- 1
Verify the gateway container is running and is not restarting repeatedly
diagnostics · containers[nestjs-core].restarts = 3
- 2
Verify the gateway can write its state directory, where the wizard persists each step
expected · The command exits 0 and prints nothing. A permission-denied error confirms this cause.
- 3
Verify the hostnames the connectivity step probes actually resolve
diagnostics · appliance.dns
- 4
Read the outbound connectivity probe results
diagnostics · appliance.egress
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 2 fails | state-dir-permissions (critical) | The wizard persists the result of each step under /var/lib/cid. When that directory is owned by root but the gateway process runs unprivileged — the state after a manual mkdir or a restore that did not preserve ownership — the write throws, and the handler surfaces it as a 500 rather than as a permissions error. The connectivity probe itself may well have succeeded.Fix: Correct the ownership of the state directory on the host so the gateway's user can write it, then restart the gateway container and retry the step. Do not make the directory world-writable; it holds appliance identity material. Verify: expected · The probe exits 0 and the wizard advances to the next step. |
| check 1 fails | gateway-crashloop (critical) | The gateway container is restarting, so a request that lands mid-restart fails with a 500 that has nothing to do with connectivity. The wizard is reporting the symptom of a boot failure — most often a refused placeholder JWT secret, or a migration that did not apply. Fix: Read the gateway container logs from the start of the most recent restart and fix the boot error itself. Check the security posture first: a placeholder JWT_SECRET refuses boot by design, and pending migrations mean the schema readers will fail.Verify: diagnostics · containers[nestjs-core].restarts = 3 If that does not apply: Attach the support bundle, which carries the gateway log tail. |
| check 3 fails | no-dns (warning) | The appliance cannot resolve the hostnames it is asked to probe. A resolver that times out rather than answering NXDOMAIN makes the step hang and then fail, which reads as a server error. Fix: Set a reachable DNS server in the appliance network configuration and re-run the step. On a DHCP-less install the resolver must be configured explicitly — see Network prerequisites. Verify: diagnostics · appliance.dns |
| check 4 fails | egress-blocked (warning) | Name resolution works but nothing gets out. DNS, TCP/TLS to a provider, and an authenticated request through the configured proxy fail independently — a successful DNS lookup is not evidence that the other two work, and treating them as interchangeable is what makes this step confusing. Fix: Read the three probe results separately in the diagnostics snapshot and fix the one that actually failed. If the site requires an outbound proxy, configure it on the appliance before retrying; an unset proxy on a proxy-only network produces a timeout, not a refusal. Verify: diagnostics · appliance.egress |
The wizard finishes and asks for setup again
The wizard's completion is written to a file, and that write is wrapped in a handler that logs and continues. An unwritable directory therefore produces a wizard that completes, unlocks the product, and locks it again at the next restart.
The setup wizard finishes and the appliance asks for setup again
You might see: 423 SETUP_REQUIRED · Appliance setup is not complete. Finish first-boot setup to use the product. · the wizard completes and the dashboard sends me back to it · every page returns 423 after setup
Before you start
Checks
- 1
Read whether the gateway considers setup complete
diagnostics · appliance.setupComplete = true
- 2
Verify the gateway can write the state directory the wizard persists into
expected · The command exits 0 and prints nothing.
- 3
Verify the gateway is not restarting between wizard steps
diagnostics · containers[nestjs-core].restarts = 3
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 2 fails | state-dir-not-writable (critical) | The wizard persists its state to a file under the shared state directory. That write is wrapped in a handler that logs and continues, so an unwritable directory produces a wizard that completes in memory, unlocks the product until the container restarts, and then locks it again with the state file still absent. The usual causes are a host directory created by hand as root, a restore that did not preserve ownership, or a read-only bind mount. Fix: Correct the ownership of the state directory on the host so the gateway's own user can write it, and confirm the mount is not read-only. Then restart the gateway container and run the wizard once more. Do not make the directory world-writable; it holds appliance identity material. Verify: expected · The probe exits 0 and the dashboard loads the product rather than the wizard. |
| check 1 fails | completion-not-persisted (critical) | The gateway reports setup as incomplete, so its setup guard answers 423 on every route outside the small allowlist the wizard, authentication and health need. Everything else in the product is locked until that state changes, which is why the dashboard sends you back. Fix: Establish why the completion was not recorded — an unwritable state directory is the common cause — then complete the wizard again. The guard is inert the moment the state records completion; no restart is needed to unlock, only to reload a lost state. Verify: diagnostics · appliance.setupComplete = true |
| check 3 fails | restart-loses-memory-state (critical) | The gateway is restarting during the wizard. Each restart reloads the state from disk, so progress that only ever existed in memory is lost and the wizard begins again from the step the file last recorded. Fix: Fix the restart first, then run the wizard once, uninterrupted. See A container keeps restarting. Verify: diagnostics · containers[nestjs-core].restarts = 3 |
Saved state does not stick
The same root cause, seen from the other direction: the setup state, the licence, update intents and the diagnostics hand-off all live under one directory, and all four fail silently when it cannot be written.
Settings the appliance saves do not survive a restart
You might see: the setup state directory is not writable · setup-state write failed · the licence I uploaded is gone after a restart · the wizard completes and comes back
Before you start
Checks
- 1
Write and remove a probe file in the shared state directory as the gateway's own user
expected · The command exits 0 and prints nothing. Permission denied or a read-only filesystem confirms this runbook.
- 2
Repeat the probe in each subdirectory the product writes
expected · All four exit 0.
- 3
Check that the filesystem holding the state directory has space
diagnostics · host.disks[/].freeGb = 2
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | ownership-or-readonly-mount (critical) | The gateway runs unprivileged and writes its durable state under one shared directory: the setup state file, the licence, update intents and the diagnostics hand-off all live there. Every one of those writes is wrapped in a handler that logs and continues, so an unwritable directory produces a product that appears to accept changes and forgets them at the next restart rather than one that reports an error. Fix: Correct the ownership of the state directory on the host so the gateway's user can write it, and confirm the bind mount is not read-only. Restart the gateway container afterwards so it re-reads whatever state exists. Do not make the directory world-writable — it holds appliance identity material. Verify: expected · The probe exits 0 and the setting survives the restart. |
| check 2 fails | subdirectory-owned-by-root (critical) | The parent directory is writable and one subdirectory is not — the state after a manual mkdir as root, or a restore that recreated the tree without preserving ownership. Only the feature that writes into that one subdirectory fails, which is why the symptom looks specific to licensing or to updates rather than to storage. Fix: Correct the ownership of every subdirectory under the state root, not just the root itself, then restart the gateway container. Verify: expected · All four exit 0. |
| check 3 fails | disk-full (critical) | The filesystem holding the state directory has no room, so writes fail for a reason that has nothing to do with permissions. The product behaves identically in both cases because both are swallowed by the same handler. Fix: Free space on the appliance, or grow the disk. See A filesystem is nearly full. Verify: diagnostics · host.disks[/].freeGb = 2 |
The providers step is empty
Provider and model rows come from the database seeder, not from code. Nothing is broken; the data was never inserted.
The setup wizard's providers step shows an empty list
You might see: GET /setup/providers returns an empty list · no providers to choose from in the wizard · provider dropdown is empty during setup · cannot select OpenAI or Anthropic in the wizard
Before you start
Checks
- 1
Verify the gateway can reach PostgreSQL — the provider list is read from it, not hard-coded
diagnostics · datastores.postgres.reachable = true
- 2
Verify no migrations are pending, so the provider and model tables exist in their current shape
diagnostics · datastores.postgres.migrationsPending
- 3
Ask the setup API for the provider list
GET /setup/providers
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 3 fails | seed-never-ran (critical) | The provider and model rows come from the database seeder, not from code. When seeding on boot is disabled the tables stay empty, and the wizard faithfully shows nothing to choose from. Nothing is broken — the data was never inserted. Fix: Enable seeding on boot, or run the seeder once against this database, then reload the wizard step. The seed inserts the provider rows and their models; it does not insert credentials, so you still supply your own API key in the next step. Verify: GET /setup/providers If that does not apply: If the seeder runs without error and the list is still empty, attach the support bundle and the seeder log. |
| check 1 fails | database-unreachable (critical) | The gateway cannot reach PostgreSQL, so every database-backed list is empty rather than erroring visibly. The providers step is simply the first place in the wizard where that becomes obvious. Fix: Start the database container and wait for its healthcheck to pass, then reload the step. If the container is running but unreachable, check that the gateway's database host, port and credentials match what the database is actually serving. Verify: diagnostics · datastores.postgres.reachable = true |
| check 2 fails | migrations-pending (critical) | The compiled migration chain contains migrations that are not recorded in the schema, so the provider or model tables are missing columns the current code selects. The query fails or returns nothing, and the wizard shows an empty list. Fix: Apply the pending migrations and restart the gateway. The entrypoint applies them on boot whenever synchronise is off, so a non-empty pending list on a booted appliance means that run failed — read the gateway log from the boot that failed rather than re-running blindly. Verify: diagnostics · datastores.postgres.migrationsPending |
Licensing
Two failures dominate activation, and they share one root cause: an appliance that demands a licence it is structurally unable to verify.
License upload fails with 400 signature
The licence file is usually fine. The appliance has no signing public key to check it against, so every licence looks forged.
License upload fails with 400 "rejected: signature"
You might see: 400 rejected: signature · 400 rejected: ENOENT · License upload → 400 · licence file will not upload
Before you start
Checks
- 1
Verify the licence trust anchor exists on the host
diagnostics · appliance.license.trustAnchorPresent = true
- 2
Verify the appliance reports an installation id
diagnostics · appliance.license.installationId
- 3
Verify the host clock is disciplined by NTP
diagnostics · host.timeSynced = true
- 4
Read the licence state the gateway reports
GET /admin/license/status → {"state":"active|trial"}
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | missing-anchor (critical) | The licence-signing public key (/etc/cid/license-pubkey.pem) was never baked into this image, or was removed. Verification has nothing to check the signature against, so every licence — including a perfectly valid one — is rejected as forged. The API surfaces this as a 400 with a signature or ENOENT message rather than as a missing-key error, which is why the licence file gets blamed. Fix: Restore the trust anchor on the host and restart the gateway container, then upload the licence again. On an appliance the anchor is part of the image: an image that lacks it was built wrong and the fix is to re-run the host repair, not to hand-place a key of unknown provenance. Follow Licensing and activation; if the appliance was built from an image that shipped without the anchor, collect the support bundle and escalate rather than sourcing a key yourself. Verify: diagnostics · appliance.license.trustAnchorPresent = true If that does not apply: Attach the support bundle. The image build, not the appliance, is at fault. |
| check 2 fails | null-installation-id (critical) | The appliance reports no installation id, so no licence can have been issued for it and no licence can bind to it. This normally means the identity was never generated during first boot, or the file holding it is not readable by the gateway container. Fix: Re-run the first-boot identity step and confirm the id appears on Settings → License. Request a licence only once a stable id is shown; a licence issued against a null id is unusable. See Licensing and activation. Verify: diagnostics · appliance.license.installationId |
| check 3 fails | clock-skew (warning) | Licence validity is bounded by dates. A host clock that is wrong by more than the licence's grace window makes a valid licence appear not-yet-valid or expired, and the error surfaces at the same place as a signature failure. Fix: Correct the host clock and enable NTP or chrony, then restart the gateway container and upload the licence again. See Network prerequisites for the NTP egress the appliance needs. Verify: diagnostics · host.timeSynced = true |
| check 4 fails | wrong-licence-for-this-box (critical) | The trust anchor, installation id and clock are all correct, so the signature check is meaningful — and it fails. The licence was issued for a different installation id, was modified in transit (a mail client re-wrapping lines is the usual culprit), or was truncated. Fix: Re-download the licence file from the issuing portal, transfer it as a binary attachment or over a checksum-verified copy, and upload it again without opening it in an editor. If the installation id in the licence does not match this appliance, request a re-issue. Verify: GET /admin/license/status → {"state":"active|trial"} If that does not apply: Attach the support bundle and the installation id shown on Settings → License. |
The appliance has no licence trust anchor
The same missing key, before anyone has tried to upload anything. With enforcement off the appliance quietly runs its trial, so the gap is often found only when the trial ends.
This appliance has no licence trust anchor and can only run the trial
You might see: No license trust anchor (/etc/cid/license-pubkey.pem) · no licence can be verified on this box · the appliance only ever runs the built-in trial · every licence file is rejected, whichever one we upload
Before you start
Checks
- 1
Read whether the licence-signing public key exists on the host
diagnostics · appliance.license.trustAnchorPresent = true
- 2
Read whether licence enforcement is switched on, which decides how bad the missing anchor is
diagnostics · appliance.license.requireLicense = false
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | anchor-absent (warning) | The licence-signing public key is part of the appliance image and lives at /etc/cid/license-pubkey.pem. Verification has nothing to check a signature against without it, so every licence — including a valid one — is rejected as forged. With enforcement off the appliance still runs its built-in trial, which is why the missing anchor can go unnoticed until the trial ends. Fix: Run the host repair from Settings → System Updates, which re-establishes the files the image should carry, then re-check the snapshot. An image that still lacks the anchor after repair was built wrong: collect the support bundle and escalate rather than placing a key of unknown provenance by hand. Verify: diagnostics · appliance.license.trustAnchorPresent = true If that does not apply: Attach the support bundle. The image build, not the appliance, is at fault. |
| check 2 fails | enforcement-on-without-anchor (critical) | Enforcement demands a valid licence and the appliance cannot verify one, so every licensed route answers 402 no matter what is uploaded. The appliance is enforcing a check it is structurally unable to pass. Fix: Restore the anchor as above and upload the licence issued for this installation id. See License upload fails with 400 signature. Turning enforcement off is a lab measure only — it disables the entitlement checks the tier relies on. Verify: GET /admin/license/status → {"state":"active|trial"} |
Every request returns 402 after setup
Licence enforcement is on and no licence resolves as valid. Administration stays reachable so the appliance can be licensed without a rescue procedure.
Every request returns 402 LICENSE_EXPIRED after the wizard completes
You might see: 402 LICENSE_EXPIRED · chat returns 402 after finishing setup · the dashboard loads but every action fails with payment required · product blocked immediately after setup
Before you start
Checks
- 1
Read whether licence enforcement is switched on for this deployment
diagnostics · appliance.license.requireLicense = true
- 2
Read the licence state the gateway resolved at boot
diagnostics · appliance.license.state = "active|trial"
- 3
Verify the licence trust anchor exists, since without it no licence can ever resolve as active
diagnostics · appliance.license.trustAnchorPresent = true
- 4
Check how long the installed licence has left
diagnostics · appliance.license.expiresAt
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 3 fails | enforcement-on-without-anchor (critical) | CID_REQUIRE_LICENSE is true and the licence-signing public key is absent. The guard demands a valid licence, and verification cannot succeed without the anchor, so every licensed route answers 402 no matter what is uploaded. The appliance is enforcing a check it cannot pass. Fix: Restore the trust anchor and upload the licence, following License upload fails with 400 signature. Turning enforcement off is a deliberate, temporary measure for a lab only — never the fix on a customer appliance, because it disables the entitlement checks the tier relies on. Verify: diagnostics · appliance.license.trustAnchorPresent = true |
| check 2 fails | no-valid-license (critical) | Enforcement is on and the licence resolved to a state other than active or trial — missing, invalid or expired. The setup wizard does not install a licence, so a freshly completed appliance with enforcement on reaches exactly this state until one is uploaded. Fix: Upload the licence issued for this appliance under Settings → License. If no licence has been issued yet, request one against the installation id shown on that page. See Licensing and activation. Verify: GET /admin/license/status → {"state":"active|trial"} |
| check 4 fails | license-genuinely-expired (critical) | A licence is installed and verifies, but its validity period has ended. Enforcement then blocks the product routes while leaving administration reachable, so the appliance can be re-licensed without a rescue procedure. Fix: Upload a renewed licence. If the expiry looks wrong, check the host clock first — a skewed clock ages a valid licence out early and produces exactly this state. Verify: diagnostics · appliance.license.state = "active|trial" |
| check 1 fails | unexpected-402-without-enforcement (warning) | Enforcement is off, so the licence guard should not be blocking anything. A 402 in this state means the value the gateway read at boot differs from the value now configured — usually the container was started before the setting changed and has not been restarted. Fix: Restart the gateway container so it re-reads its configuration, then retry. If 402 persists with enforcement off, collect the support bundle: the effective configuration and the configured one disagree. Verify: diagnostics · appliance.license.requireLicense = false If that does not apply: Attach the support bundle, including the effective posture section. |
The licence has expired
An appliance that worked and now blocks its product routes. Check the clock before requesting a renewal: a skewed clock ages a valid licence out early and produces exactly this state.
The licence has expired and licensed endpoints answer 402
You might see: License expired / product locked · 402 LICENSE_EXPIRED · License state is "expired" · the trial ran out
Before you start
Checks
- 1
Read the licence state the gateway resolved
diagnostics · appliance.license.state = "active|trial"
- 2
Verify the host clock is disciplined, because a skewed clock ages a valid licence out early
diagnostics · host.timeSynced = true
- 3
Verify the trust anchor exists, since without it no renewal can ever verify either
diagnostics · appliance.license.trustAnchorPresent = true
- 4
Read the licence status the API reports
GET /admin/license/status → {"state":"active|trial"}
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 or 4 fails | expired-or-invalid (critical) | A licence is installed and its validity period has ended, or it no longer verifies. The guard then blocks the licensed routes — chat, detection, inspection, image and document analysis — with 402 while leaving administration reachable, so the appliance can be re-licensed without a rescue procedure. Fix: Upload a renewed licence under Settings → License. Request the renewal against the installation id shown on that page: a licence issued for a different id will not verify here. See Licensing and activation. Verify: diagnostics · appliance.license.state = "active|trial" |
| check 2 fails | clock-skew-fake-expiry (warning) | Licence validity is bounded by dates read from the host clock. A clock that is wrong by more than the grace window makes a current licence look expired, and the product blocks exactly as it would for a real expiry. Fix: Correct the host clock and enable time synchronisation, then restart the gateway container and re-read the licence state before requesting a renewal. See The host clock is not synchronised. Verify: diagnostics · host.timeSynced = true |
| check 3 fails | renewal-cannot-verify (critical) | The trust anchor is absent, so the renewal you are about to upload will be rejected for the same reason the current licence no longer verifies. Uploading first wastes the maintenance window. Fix: Restore the trust anchor before uploading anything. See This appliance has no licence trust anchor. Verify: diagnostics · appliance.license.trustAnchorPresent = true |
TLS
Nothing is listening on 443
A refused connection is not a certificate error, and the two share a cause more often than you would expect: the front end refuses to start with an unusable certificate rather than serving a broken one.
Nothing is listening on 443
You might see: Nothing is listening on 443 · connection refused on 443 · https://<ip>/ refuses the connection · the dashboard cannot be reached over HTTPS
Before you start
Checks
- 1
Read what holds port 443 on the appliance host
diagnostics · appliance.tls.servedBy
- 2
Verify the appliance has a certificate to serve at all
diagnostics · appliance.tls.mode = "none"
- 3
Verify the dashboard container the listener proxies to is running
diagnostics · containers[frontend].state = "running"
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | listener-absent (critical) | No process holds 443, so the client's connection is refused before any handshake begins and the browser reports a connection error rather than a certificate error. On a freshly installed appliance the usual reason is that the HTTPS front end never installed; on a running one it follows a failed certificate rollout, because the front end refuses to start with an unusable certificate rather than serving a broken one. Fix: On a fresh install, re-run the HTTPS installer on the host with sudo bash /opt/cid-install/install-https.sh. On a running appliance, run the host repair from Settings → System Updates, which re-establishes the front end and re-applies the certificate intent. If repair reports a certificate problem, fix that first — the front end will keep refusing to start until the certificate it is told to serve is valid.Verify: diagnostics · appliance.tls.servedBy If that does not apply: Attach the support bundle, which carries the host HTTPS inventory. |
| check 2 fails | no-certificate-to-serve (critical) | The appliance has no certificate configured — neither one issued by its own CA nor an uploaded one — so there is nothing for a listener to present and nothing can be served on 443. Fix: Issue a certificate from the appliance CA under Settings → Certificates, or upload one your organisation's CA signed against a signing request generated there. See Naming, DNS and TLS. Verify: diagnostics · appliance.tls.mode = "none" |
| check 3 fails | frontend-down (critical) | The listener terminates TLS and proxies to the dashboard container. With that container down, 443 either refuses connections or answers with an upstream error, depending on which half failed first. Fix: Start the dashboard container and read its log if it exits again. See A container keeps restarting. Verify: diagnostics · containers[frontend].state = "running" |
The certificate is expiring or has expired
Browsers refuse an expired certificate outright, and the clients configured to trust this appliance fail closed rather than degrading — so enforcement stops at the same moment the dashboard becomes unreachable.
The appliance HTTPS certificate is expiring or has expired
You might see: The appliance HTTPS certificate expires in 6 day(s) · ERR_CERT_DATE_INVALID · certificate expired · your connection is not private
Before you start
Checks
- 1
Read how many days the served certificate has left
diagnostics · appliance.tls.daysLeft = 0
- 2
Check whether the certificate has entered the renewal window — 30 days, the `CID_TLS_RENEW_DAYS` default in `appliance/tls/ensure-tls.sh`
diagnostics · appliance.tls.daysLeft = 30
- 3
Read which kind of certificate the appliance serves, because the renewal path differs
diagnostics · appliance.tls.mode = "unknown"
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | already-expired (critical) | The served certificate is past its expiry. Browsers refuse the connection, and the clients that were configured to trust this appliance — the browser extension, the endpoint agent, the inline proxy's callers — fail closed rather than degrading, so enforcement stops at the same moment the dashboard becomes unreachable. Fix: Issue a replacement now. A certificate issued by the appliance's own CA is reissued in place from Settings → Certificates; an externally signed one needs a fresh signing request, a signature and an upload. See Naming, DNS and TLS. Verify: diagnostics · appliance.tls.daysLeft = 0 |
| check 2 fails | inside-renewal-window (warning) | The certificate is still valid and inside the window where renewal should start. An externally signed certificate needs the signing turnaround as well as the appliance work, and that turnaround, not the appliance, is the long pole. Fix: Renew now rather than at expiry. Generate the signing request from Settings → Certificates, have it signed, and upload the result. An appliance-CA certificate is reissued in place with no external dependency. Verify: diagnostics · appliance.tls.daysLeft = 30 |
| check 3 fails | mode-unknown (warning) | The appliance cannot say which certificate it serves, so neither renewal path can be recommended and the expiry reading may be about a certificate that is no longer the one on 443. This is what the snapshot reports when the host TLS inventory is missing. Fix: Run the host repair from Settings → System Updates so the TLS inventory is written again, then re-read the snapshot before renewing anything. Verify: diagnostics · appliance.tls.mode = "unknown" |
Updates
The appliance cannot say which release it runs
The version comes from files the image build and the updater write. When none can be read, the version falls back to a placeholder and every comparison against the channel is meaningless.
The appliance cannot say which release it is running
You might see: This box cannot say which release it is running · System Updates shows version 0.0.0 · current version 0.1.0 on a box that is not 0.1.0 · the update page offers nothing and shows no version
Before you start
Checks
- 1
Read the product version the appliance reports
diagnostics · appliance.version = "0.1.0"
- 2
Read the host-repair status, which is what rewrites the version stamp
GET /admin/system-update/host-repair
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | no-version-stamp (warning) | The appliance reads its installed version from files the image build and the updater write. When none of them can be read the version falls back to a placeholder, and every comparison against the channel manifest is then meaningless — which is why the page can show a version that is obviously wrong and simultaneously refuse to offer an update. Support answers are guesses until this is fixed. Fix: Run the host repair from Settings → System Updates, which rewrites the appliance version file, then reload the page. A placeholder that survives host repair means the image was built without its version stamp. Verify: diagnostics · appliance.version = "0.1.0" If that does not apply: Attach the support bundle and the version named in the installer's boot menu entry. |
| check 2 fails | host-agent-absent (warning) | The host-repair endpoint cannot answer because the host agent has never written its inventory, so the update page has no host-side data to render and nothing can rewrite the version file. On a non-appliance deployment there is no host agent at all and this page is not the supported upgrade path. Fix: Run the host agent's job on the appliance and reload the page. If this is a compose deployment rather than an appliance, update it by pulling the release images instead. Verify: GET /admin/system-update/host-repair |
The update channel is unreachable
Expected and harmless on a deliberately air-gapped appliance, which updates from a signed offline bundle. On a connected one it is an egress problem wearing an update-shaped mask.
The update channel is not reachable from the appliance
You might see: The update channel host is not reachable from inside the gateway · Check for updates does nothing · update check returns 500 · the appliance never finds a new version
Before you start
Checks
- 1
Verify an update channel is configured at all
diagnostics · appliance.channel
- 2
Read whether the channel answered on the last attempt
diagnostics · appliance.update.channelReachable = true
- 3
Read the name-resolution results for the hosts the appliance must reach
diagnostics · appliance.dns
- 4
Read whether an outbound proxy is configured, on a network that requires one
diagnostics · appliance.proxy.configured = true
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | channel-unset (warning) | No release channel is configured, so the update check has no manifest to fetch. The API rejects the call, and the page shows a button that fails rather than a prompt to configure anything. Fix: Set the update channel under Settings → System Updates. On an air-gapped appliance, choose the offline channel and upload a signed bundle instead of pointing at a URL. Verify: diagnostics · appliance.channel |
| check 2 fails | channel-blocked (warning) | The channel is configured and the fetch failed. On an appliance that is deliberately air-gapped this is the expected state and needs no action; on a connected one it means the HTTPS path to the channel host is blocked, intercepted, or requires a proxy the appliance does not have. Fix: Read the egress results in the diagnostics snapshot and fix the test that failed rather than the one you assume failed. See The appliance cannot get out. If the appliance is meant to stay offline, switch it to the offline channel so the failure stops being reported as a fault. Verify: diagnostics · appliance.update.channelReachable = true |
| check 3 fails | dns-cannot-resolve-channel (warning) | The appliance cannot resolve the channel host, so the fetch never reaches the network. DNS succeeding elsewhere proves nothing: the appliance uses this network's resolver, not the browser's. Fix: Set a resolver that answers for the channel host and re-run the check. See Network prerequisites. Verify: diagnostics · appliance.dns |
| check 4 fails | proxy-missing (warning) | This site sends everything through a proxy and the appliance has none configured, so the fetch is attempted directly and times out. A timeout looks like a dead channel rather than a missing setting. Fix: Configure the outbound proxy on the appliance and restart the gateway container so it re-reads the setting. Verify: diagnostics · appliance.proxy.configured = true |
Any phase
The host clock is not synchronised
Licence expiry, certificate validity, token lifetimes and every detection timestamp are decided by this clock. A drifting one produces three failures that each look like something else.
The host clock is not synchronised
You might see: The host clock is not synchronised · clock is not synchronised — if the registry login fails on a certificate error, this is why · a valid licence is reported as expired · tokens are rejected as expired immediately after sign-in
Before you start
Checks
- 1
Read whether the host clock is disciplined by a time service
diagnostics · host.timeSynced = true
- 2
Verify the appliance can reach a time source at all
expected · The status reports the clock as synchronised, or names a server the network permits.
- 3
Check whether the skew has already aged the licence out
diagnostics · appliance.license.state = "active|trial"
Causes and fixes
| Reached when | Cause | Explanation, fix and verification |
|---|---|---|
| check 1 fails | no-time-source (warning) | Licence expiry, certificate validity, session-token lifetimes and every detection timestamp are decided by this clock. A drifting clock therefore expires licences early, rejects tokens that were just issued, and makes a current certificate look not-yet-valid — three failures that each look like something else. During an install it also breaks the registry login with a certificate error that reads like a network fault. Fix: Enable time synchronisation on the appliance host and point it at a time source this network permits. Restart the gateway container afterwards so anything it cached at boot is re-evaluated against the corrected clock. Verify: diagnostics · host.timeSynced = true |
| check 2 fails | ntp-blocked (warning) | A time client that cannot reach a server leaves the clock free-running, and the failure is silent: nothing on the appliance reports that time is wrong until something dated fails. Many sites block outbound time traffic and provide an internal server instead. Fix: Point the appliance at the site's own time server, or open the time protocol to a permitted external one. See Network prerequisites. Verify: expected · The clock reports as synchronised. |
| check 3 fails | licence-mis-aged (critical) | The skew has already pushed the licence out of its validity window, so the product blocks exactly as it would for a real expiry. Requesting a renewal at this point produces a licence that will look expired too. Fix: Correct the clock first, then re-read the licence state. Request a renewal only if the state is still expired once time is right. See The licence has expired. Verify: diagnostics · appliance.license.state = "active|trial" |
If none of these match
- Read the diagnostic snapshot end to end; its findings section names a runbook for every rule that fired.
- Check the health matrix — a service that answers and reports itself unhealthy usually means a model that did not load.
- Collect the support bundle and escalate, with the installation id and the appliance version.
Last updated on