Bahriya Tersane

Documentation / compliance

Compliance and Assurance Convoy

Evidences a regulatory regime’s technical controls in a Kubernetes cluster.

No regime’s content is in the chart. Each regime’s evidence model is an image of its own, 1x.ax/tersane/convoy/compliance/regimes/<regime>/catalogue, and the chart holds only its reference, pinned by digest, in files/regimes.json. The chart is granted with the Convoy and each regime is granted on its own, so a regime the licence does not grant cannot be pulled, and stops alone. Activating a regime describes how a regime is added.

Prerequisites

Kubernetes 1.36 or later, which the Bahriya Kubernetes Engine provides from release 2.2: each regime’s evidence model is mounted as an image volume, and the container runtime must support image volumes.

The Bahriya Tersane kyverno and trivy-operator charts, installed separately. The chart refuses to install without them. To render it outside a cluster, pass their APIs:

helm template compliance . \
  --api-versions policies.kyverno.io/v1/ImageValidatingPolicy \
  --api-versions aquasecurity.github.io/v1alpha1/ClusterComplianceReport

For the Compliance App, an OpenID Connect identity provider that the app can reach, directly or through an HTTP proxy, or local accounts held in a Kubernetes Secret.

Installation

Install the three charts in this order: Kyverno, then trivy-operator, then the Convoy. The Convoy refuses to install until the other two are present. The chart repositories and the credentials are described in Access and credentials.

  1. Install Kyverno and trivy-operator, as their pages describe: kyverno and trivy-operator. If you will assess only some namespaces, set each engine’s namespaces now, as Keeping the engines inside the scope describes.

  2. Create the release’s namespace and its pull secret. The examples use the release name compliance in the namespace compliance. 1x-ax.json is the credential file written as Creating the pull secret describes, which keeps the token out of your shell history; keep it until step 5:

    kubectl create namespace compliance
    kubectl --namespace compliance create secret generic 1x-ax \
      --type=kubernetes.io/dockerconfigjson \
      --from-file=.dockerconfigjson=1x-ax.json
    

    Do not create the node collector’s namespace (compliance-node): the chart creates it, and the install stops if it already exists.

  3. Write a values file with at least the regimes your licence grants:

    regimes:
      - nca-ecc
    

    The Parameters list every other setting, such as scope.namespaces, the schedule, email and object storage.

  4. Install the chart:

    helm install compliance compliance-convoy/compliance \
      --namespace compliance --version <chart version> -f <values file>
    
  5. Create the pull secret in the node collector’s namespace, which the install has just created:

    kubectl --namespace compliance-node create secret generic 1x-ax \
      --type=kubernetes.io/dockerconfigjson \
      --from-file=.dockerconfigjson=1x-ax.json
    

    Then delete 1x-ax.json. Until this secret exists, the node collector’s Jobs cannot pull their image, and a run records the node facts as not read.

  6. Start a first run and read its report, as step 5 of Activating a regime shows.

Regimes

Regime Value Catalogue Mapping to NIST SP 800-53
NCA Essential Cybersecurity Controls, ECC-2:2024 nca-ecc 176 requirements Mamluk’s own
Qatar Central Bank, Technology Risks (January 2018) qcb 516 requirements Mamluk’s own
UAE Information Assurance Standard 2.1 (November 2025) uae-ia 134 requirements The standard’s own, from its Annex D
National Information Assurance Standard, version 2.1, issued by the National Cyber Security Agency (May 2023) qatar-nia 356 requirements Mamluk’s own
SAMA Cyber Security Framework, version 1.0 (May 2017) sama-csf 409 requirements Mamluk’s own
Central Bank of the UAE, Operational Risk Management Regulation cbuae-orm 63 requirements Mamluk’s own
Monetary Authority of Singapore, Technology Risk Management: Notices FSM-N05 and FSM-N06 (May 2024) and Guidelines (January 2021) mas-trm 259 requirements: 15 from the notices, binding, and 244 from the Guidelines, guidance Mamluk’s own
Central Bank of Iraq, Cyber Resilience Controls, first edition (June 2024), and Banking Reform Standards B2, B3, B7 and B9 (2025) cbi-crc 524 requirements: 434 from the Controls, for all banks and payment providers, and 90 from the Standards, for private banks Mamluk’s own

Each report states whose mapping of the regime to NIST SP 800-53 it uses. Where the regulator publishes none, the mapping is Mamluk’s own.

MAS Technology Risk Management (mas-trm). One regime holds the two notices and the Guidelines. Each requirement shows its legal force: a notice’s paragraph reads “Notice (binding)”, with the notice’s number and a space before its ID (for example N06 4.6); a Guideline paragraph reads “Guidelines (guidance)”, with the Guidelines’ own paragraph number. In OSCAL the same distinction is the applicability property, always for a notice and risk for a Guideline paragraph. The notices to licensed insurers (FSM-N03, FSM-N04) and to capital markets intermediaries (FSM-N21, FSM-N22) state the same requirements; the report uses the bank notices’ numbering for all.

Central Bank of Iraq (cbi-crc). One regime holds the Cyber Resilience Controls and the technology standards B2 (core banking), B3 (online banking), B7 (infrastructure and data) and B9 (business and operational resilience). Each requirement shows the text it comes from: “Controls (all banks and payment providers)” or “Standards (private banks)”; in OSCAL the applicability property is always for a Controls requirement and risk for a Standards requirement. The Controls are published in Arabic only, and the Standards in Arabic with an English reference translation; the summaries in the report are Mamluk’s own English paraphrase. A Controls ID carries a function prefix (for example PR-9.1a) and a Standards ID the standard and item (for example B7 B.6); the prefixes and the letters added to the CBI’s numbers are Mamluk’s. The data centre and disaster recovery requirements (in Iraq, the recovery site at least 75 km away) are the bank’s to evidence. Evidence stays on the cluster’s volume unless delivery by S3 or by email is configured; an endpoint, mailbox or mail relay outside Iraq places the evidence outside Iraq, so set s3.endpoint to a store in Iraq.

Regimes and their authorities

Each regime’s report, PDF, email, OSCAL document and the Compliance App carry the statement below on the regime’s authority, followed by the regime’s notes on its texts: who owns them, the language they are published in, that the requirement summaries are Mamluk’s paraphrase, and whose mapping to NIST SP 800-53 the report uses. In the OSCAL document both are in the metadata remarks, and each is also a back-matter resource of its own, selected by the property resource (in the namespace tersane.bahriya.cloud/ns/oscal, which the document writes with the https scheme) with the value statement or regime-notes. The Compliance App shows the notes on each run’s page.

Regime Authority Statement
nca-ecc National Cybersecurity Authority (NCA), Kingdom of Saudi Arabia Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the National Cybersecurity Authority (Saudi Arabia).
qcb Qatar Central Bank (QCB), State of Qatar Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by Qatar Central Bank.
uae-ia UAE Cyber Security Council, United Arab Emirates Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the UAE Cyber Security Council.
qatar-nia National Cyber Security Agency (NCSA), State of Qatar Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the National Cyber Security Agency (Qatar).
sama-csf Saudi Central Bank (SAMA), Kingdom of Saudi Arabia Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Saudi Central Bank.
cbuae-orm Central Bank of the UAE, United Arab Emirates Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Central Bank of the United Arab Emirates.
mas-trm Monetary Authority of Singapore (MAS), Republic of Singapore Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Monetary Authority of Singapore.
cbi-crc Central Bank of Iraq (CBI), Republic of Iraq Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Central Bank of Iraq.

Corrections to earlier reports

Multi-factor authentication as the organisation’s evidence (this chart release). Requirements that ask for multi-factor authentication are now the organisation’s to evidence. Earlier, NCA ECC 2-2-3-2, Qatar NIA AM 36, QCB 8.10.5.2 and SAMA CSF 3.3.5.4.e, 3.3.5.4.f.1.a and 3.3.5.4.f.1.b could show Pass, with partial coverage, when sign-in to the cluster was delegated to the organisation’s identity provider, although the Convoy does not test whether that provider requires a second factor. They now show the organisation’s evidence, with the reason. QCB 8.10.2.7 and UAE IA T5.2.3 keep their verdicts on the authentication and access controls the cluster does evidence, and leave the second factor to the organisation, as their scopes state. If an assessor has read an earlier report, its Pass for those six requirements should be read as evidence that sign-in is delegated to the identity provider, and nothing more.

The audit log directory as a check of its own (an earlier chart release). The collector of that release changed what the following show.

  • No Pass without the audit log’s evidence. Earlier, the node file permissions check also judged the API server’s audit log directory, and passed on its other files where that directory could not be examined. A requirement resting on it could then show Pass although the audit log was never looked at. The directory is now judged by the audit log permissions check, which is Not evaluated, with the reason, wherever the directory is not examined. See Node facts.
  • Requirements that now rest on the audit log’s protection. QCB 8.4.4 (logs protected from change or deletion) and UAE IA T3.2.2 (critical logs protected) gain the check. QCB 8.4.14 and 9.1.1.10, and SAMA CSF 3.3.14.4.h and 3.3.15.4.g, which ask for logs to be protected, were evidenced by the organisation alone; the cluster now evidences the protection of the API server’s audit log, and the organisation the rest, as each scope states. Each can now Fail where the audit log is open to other users, or be Incomplete where it cannot be examined, including on a Bahriya Kubernetes Engine release that does not yet set these modes, where upgrading resolves it.
  • The hardening requirements no longer judge the audit log. NCA ECC 1-6-2-2, 1-6-3-5 and 2-3-2, QCB 9.1.1.8, UAE IA T3.1.1 and SAMA CSF 3.3.8.6.a are judged on the Kubernetes configuration, credentials and keys as before, without the audit log directory, so its modes no longer make them fail.
  • NCA ECC and CBUAE. Neither names a requirement for the protection of logs in the scope the Convoy assesses, so neither regime judges the audit log’s protection. Their other audit log checks, logging and its retention, are unchanged.
  • Runs written before that release read as before.

What each check establishes, and node facts in a scoped run (an earlier chart release). The collector and the Compliance App of that release changed what the following show.

  • What each check establishes. Each check in results.json, the report and the Compliance App’s requirement screen now carries a sentence saying what it establishes, from the regime’s evidence model. Each run also records, as run.model, the SHA-256 of the model it judged from, so a stored run’s wording and verdicts can be traced to the exact model. No verdict changes. Runs written before this release read as before, without the sentence.
  • Node facts in a scoped run. In a scoped run that does not list kube-system, the node checks that need nothing from kube-system are now judged, where earlier every node check was Not evaluated. The etcd data directory check is Not evaluated, and each requirement evidenced by the node file permissions check says beside its result that the audit log directory was not examined. A run of the whole cluster, or one that lists kube-system, is judged as before. See Scoped assessment.

Scoped assessment, segmentation in meshed namespaces, the audit log directory and the outcome value in OSCAL (an earlier chart release). The collector and the Compliance App of that release changed what the following show.

  • Scoped assessment. The Convoy can assess only the namespaces you list. Existing installations assess the whole cluster, as before. See Scoped assessment.
  • Organisational evidence in OSCAL. In the OSCAL output, a requirement the organisation evidences now has the outcome organisational-evidence, as its outcome property and its status reason. Earlier output gave customer. Tooling that matches on that value needs the new one. results.json keeps the value customer, so runs written before this release read as before. The emailed summary counts these requirements as “organisational evidence”.
  • Audit. A policy’s action may be Audit: the result is recorded, and no warning is returned to whoever applied the object.
  • Segmentation in meshed namespaces. The namespace NetworkPolicy check no longer passes for a namespace in a service mesh whose traffic can be pooled, where a NetworkPolicy change may not end connections already open. Such a namespace is now Not evaluated, so a requirement that passed through it is now Incomplete: NCA ECC 2-5-2, 2-5-3-1, 2-5-3-2 and 2-5-3-5; QCB 8.1.7, 8.10.1.9, 9.1.1.3, 9.1.1.7, 9.7.1.2 and 9.7.2.1; UAE IA T4.1.1 and T4.1.3; Qatar NIA NS 5, SS 4, SS 21 and SS 23; and SAMA CSF 3.3.7.4.g and 3.3.8.6.e. A cluster with no mesh is judged as before. See NetworkPolicies.
  • The audit log directory. The node file permissions check now also judges, on each control-plane node, the API server’s audit log directory and the files in it. A cluster where they are open to other users now Fails that check. See Node facts.

Organisational evidence in the app, and three corrections (an earlier chart release). The collector and the Compliance App of that release changed what the following show.

  • Audit-log retention. The audit-log retention check no longer passes when a file-count limit can remove logs sooner than the regime’s period. Such clusters now show Not evaluated, with the central log store as the evidence. The API server deletes a rotated log at its age limit or its file-count limit, whichever comes first, so the age limit alone is no retention period. A cluster whose age limit is below the period still Fails.
  • NCA ECC cryptography. NCA ECC 2-8-2, 2-8-3-1 and 2-8-3-3 are now reported as organisational controls, because a platform check cannot yet confirm that only NCS-approved algorithms are negotiated, in transit and at rest. The kubelet cipher check’s fix now names AES-GCM suites only.
  • When an assessment ended. Each assessment result in the OSCAL output now ends when the run ended. Earlier output gave the start time as the end.
  • Organisational evidence in the app. For each organisational or partly covered requirement, the app keeps comments, attached files, a named owner and a review date, and shows a notice when a review date has passed or a regime’s newest run is overdue. Files are scanned on upload; evidence never changes a verdict; the signed export carries every file. See Organisational evidence in the Compliance App.
  • Each regime’s statement. The report, the PDF, the email, the OSCAL output, the app’s screens and its signed export carry the regime’s statement on its authority, shown above.

Proof that logs arrive, and the control plane read directly (an earlier chart release). The collector of that release changed what the following checks showed.

  • Log forwarding passes only when the records arrive: Fluent Bit’s own counters for its outputs to a central store show records delivered since the last run, with no error, failed retry or dropped record. Earlier reports passed a configured output whether or not anything reached the store. A delivery that fails, or an output that delivered nothing, now Fails, naming the Fluent Bit Pod; where the counters cannot be read, the check is Not evaluated, and says why. The collector reads the counters directly from each Fluent Bit Pod on its HTTP server’s port, which its NetworkPolicy admits, and is granted no Pod proxy.
  • Platform settings on a cluster whose control plane runs as static Pods, as kubeadm and kind run it, and as the Bahriya Kubernetes Engine did before release 2.3: audit logging and its retention, anonymous access, encryption of Secrets at rest and the control-plane TLS settings are read from the API server’s own settings, and judged. A setting that is off is Not provided. Earlier reports left them to the organisation. The other platform settings, and every platform setting on a managed control plane such as EKS, AKS or GKE, remain organisational evidence.
  • The Compliance App stops when a database migration meets an object the migrations did not make, as before, and now finds it from the migration’s own failure rather than from the wording of a warning.

A report everyone reads the same way (an earlier chart release). The collector and the Compliance App of that release changed what the following requirements show.

  • Organisational evidence. Requirements the organisation evidences outside the cluster are now called Organisational evidence. Earlier reports called them Customer evidence; runs store them under the same key, customer, so earlier runs and OSCAL output read as before (OSCAL output names them organisational-evidence from a later release, above). Organisational requirements show no coverage, which applies only to what the cluster evidences.
  • Nothing to check. A new outcome, for a cluster that has nothing of the kind a check judges outside the exempt namespaces yet: no application workload, namespace or Ingress, or no backed-up volume. It is neither a pass nor a failure. Earlier reports showed these as Not evaluated.
  • Platform requirements on a cluster that declares no platform settings. On a cluster that is not a Bahriya Kubernetes Engine cluster, and on one before release 2.3, the platform requirements are Organisational evidence, technical and outside the cluster: the control-plane settings are evidenced by the cloud provider, or whoever runs the control plane, for example with an attestation or a configuration export. The report gives the reason. Earlier reports showed them as Not evaluated. Where the platform settings could not be read on a BKE cluster, they are still Not evaluated.
  • Backups. Velero is judged as Longhorn is: an available backup storage location and an active schedule. Backups pass when either is configured to back up. A cluster with neither Longhorn nor Velero now Fails; earlier reports showed Not evaluated.
  • Log forwarding. A cluster with no log shipper now Fails. Vector, Fluentd, Promtail, Grafana Alloy, Filebeat, the OpenTelemetry Collector, the Datadog Agent, and Fluent Bit outside logging, are named, and the check is Recorded: where they send logs is the organisation’s to evidence.
  • The namespace default is judged by the policies only while it holds a workload, and netdata, a namespace of the Bahriya Kubernetes Engine, is exempt.
  • NIST SP 800-53 controls by name. Each control shows its title, and the report lists NIST’s statement of each with a link to NIST. QCB 8.2.2 named IR-10 and UAE IA T7.4.2 named SA-15(9), both of which NIST has withdrawn; they now name IR-4(11) and SA-3(2), which NIST incorporated them into. Both are organisational, so no outcome changes.

Images not scanned (an earlier chart release). The collector of that release changed the vulnerability and operating-system checks, and the requirements that follow them.

  • The vulnerability and operating-system checks are now Not evaluated while any workload outside the exempt namespaces has not been scanned by trivy-operator, and they name those workloads. A failure among the images that were scanned still fails. Earlier reports passed them on the workloads that had been scanned, so NCA ECC 2-3-3-3 and QCB 8.3.2 and 9.1.14.7, which name the vulnerability check without the scan freshness check, could show Pass while the scanner had stopped or had not yet reached the cluster’s workloads. A cluster with no workload outside the exempt namespaces passes neither check either.

Three corrections to the collector (an earlier chart release). The outcomes of the following change; every other outcome is as before.

  • Log forwarding on a Bahriya Kubernetes Engine cluster. The previous collector read Fluent Bit’s configuration only from a ConfigMap named fluent-bit. The Bahriya Kubernetes Engine names it bke-fluent-bit, after its Helm release, so on its clusters the check said Fluent Bit was not installed. The collector now reads the ConfigMap that Fluent Bit’s DaemonSet in logging mounts, and may read any name in fluentBit.configMaps (bke-fluent-bit and fluent-bit by default). QCB 8.4.10 and UAE IA T3.2.3 follow the check.
  • Operating-system support with nothing to judge. Where no image outside the exempt namespaces runs on an operating system trivy recognises, the check passed with nothing judged. It is now Not evaluated, as documented. QCB 8.8.4 follows the check.
  • Longhorn or Fluent Bit absent. Kubernetes refuses a read the collector has no rights for before it says whether the object exists, and the chart grants those rights only in namespaces that exist. On a cluster without Longhorn or Fluent Bit, the checks therefore said their configuration “was not read in this run”, as if a read had failed. They now say the tool is not installed, decided from the API’s discovery document and the namespace list. A refusal where the tool is installed is still reported as a fault.

A review of every requirement’s class (an earlier chart release). Each requirement of NCA ECC, QCB and UAE IA was read again against the checks its NIST controls reach. The outcomes of the following change; every other outcome is as before.

  • QCB 9.1.13.2 to 9.1.13.5 concern virtual machines, hypervisors and their management. Earlier reports judged them by the cluster’s own hardening, access control, patching and audit log, which say nothing of the virtualisation layer beneath it. They are now Organisational evidence. The cluster’s own results remain under QCB 9.1.1.8, 8.10.1.2, QCB 8.3.2 and QCB 8.4.4.
  • QCB 9.1.1.6 (default accounts disabled) now has a verdict: whether anonymous requests are refused and whether Pods receive the default service account’s token. Earlier reports showed it as Organisational evidence.
  • QCB 8.10.4.2 now takes its verdict on “logged” from the API server’s audit log. Earlier reports showed it as Recorded.
  • UAE IA T3.1.8 (software installed only by authorised administrators) now has a verdict: whether the cluster runs only images from the allowed registries, signed by a configured key. Earlier reports showed it as Organisational evidence.
  • NCA ECC 1-9-5 (access rights reviewed when employment ends) now records the cluster’s role bindings for the review. Earlier reports showed it as Organisational evidence.
  • Seven requirements on backups and on keeping logs centrally are now judged by the new checks below: NCA ECC 2-9-2 and 2-9-3-1, QCB 8.7.2, QCB 8.7.3 and QCB 8.4.10, and UAE IA T3.1.7 and UAE IA T3.2.3.

If an assessor has read an earlier report, its results for QCB 9.1.13.2 to 9.1.13.5 should be read as evidence about the cluster, not about the virtualisation layer, which remains the bank’s to evidence.

QCB 8.3.6, QCB 8.4.8, QCB 8.8.4, 8.10.1.6, 8.10.4.4 and 9.1.1.5. Reports made before this correction showed QCB 8.4.8, QCB 8.8.4, 8.10.1.6, 8.10.4.4 and 9.1.1.5 as Pass in error, and judged QCB 8.3.6 by whether any vulnerability was open. Each now shows Recorded.

  • QCB 8.4.8 asks for a six-monthly review of user access, 8.10.1.6 for a periodic audit of accounts against current roles, and 8.10.4.4 for a current list of privileged users. The report records the cluster’s role bindings for each, but no rule can judge from them whether the review took place. The Pass came from a different check: that people sign in through an identity provider. That is true, but it does not evidence a review.
  • QCB 8.3.6 asks for the delay in patching to be measured against the bank’s baseline. The report now records the age of each open vulnerability, and no longer gives a verdict from whether any vulnerability is open.
  • QCB 8.8.4 asks for end-of-life and end-of-support dates to be tracked. The Pass came from the scans of the workloads’ images, which judge whether critical or high vulnerabilities are open, not whether an image’s operating system is still supported. The report now records the versions of the Bahriya Kubernetes Engine and Kubernetes that the cluster runs, and no check judges support. Two checks now judge it; see End-of-support checks.
  • 9.1.1.5 asks for infrastructure and network devices to be tested for vulnerabilities. The Pass came from the scans of the workloads’ images, which do not reach the control plane, the platform or network devices. The report now records the versions of the Bahriya Kubernetes Engine and Kubernetes that the cluster runs, and its inventory.

If an assessor has read an earlier report, its Pass for QCB 8.4.8, 8.10.1.6 and 8.10.4.4 should be read as evidence that people sign in through the organisation’s identity provider, and its Pass for QCB 8.8.4 and 9.1.1.5 as evidence that the workloads’ images carry no open critical or high vulnerability, and nothing more. The review, the testing and the tracking of support dates remain the bank’s to evidence.

At the same time, three requirements on multi-factor authentication (NCA 2-2-3-2, QCB 8.10.2.7 and 8.10.5.2) gained a scope. It states that the evidence is the delegation of sign-in to the identity provider, and that whether that provider requires a second factor is the organisation’s to evidence. Eight requirements whose verdict comes from the workloads but which name systems, devices, databases or assets beyond them (NCA 2-1-5 and 2-3-3-3; QCB 5.1.4, QCB 8.3.2, QCB 8.8.4, 9.1.13.4, 9.1.14.7 and 9.4.2.3) gained a scope saying what the verdict covers. Four requirements on identification and remote access (QCB 8.10.2.1, 8.10.2.2, 8.10.5.3 and 9.10.1.9) gained a scope saying that the verdict concerns sign-in through the identity provider, and that shared credentials such as kubeadm’s admin.conf, and SSH access to the nodes, are the bank’s to evidence. Their outcomes are unchanged.

End-of-support checks

QCB 8.8.4 carries a verdict again, from two checks:

  • Platform releases. The Kubernetes release and the Bahriya Kubernetes Engine release that the cluster runs are compared with their published support periods. A release out of support fails.
  • Image operating systems. Each workload image’s operating system, as trivy-operator’s vulnerability scans record it, fails if it is past its end of service life. Images with no operating system that trivy recognises (scratch and static images, and operating systems trivy does not know) do not block the check. The report lists each of them by name, so that they can be reviewed by hand. Their packages remain judged by the vulnerability check.

QCB 8.8.4 shows Incomplete only where trivy-operator has reported no vulnerability scan. No new scan is needed after the upgrade, because the existing scans already record each image’s operating system. NCA ECC results are unchanged.

Backup and log forwarding checks

Three checks read the cluster’s backups and its logging:

  • Backups. Longhorn has a backup target that it reports available, and a recurring backup job, such as the bke-daily-backup job that the Bahriya Kubernetes Engine creates; or Velero has a backup storage location that it reports Available, and a schedule that is enabled and not paused. Either is enough. The last backup of each volume and each schedule, and its age, are recorded beside the verdict. With neither tool installed, the check Fails.
  • Backup encryption. Every backed-up Longhorn volume outside the exempt namespaces is encrypted, so that its backups leave the site encrypted. The encrypted StorageClasses are listed beside the verdict. Velero’s backups are encrypted, or not, by the object store it writes to, which the cluster does not show: with Velero alone the check is Not evaluated, and records which server-side encryption settings each storage location names. With no backup tool, nothing leaves the site, and there is Nothing to check.
  • Log forwarding. An output of Fluent Bit’s configuration sends to a host, or an s3 output to a bucket, and the records arrive: Fluent Bit’s own counters for those outputs, read on each of its Pods, show records delivered since the last run, or since the Pod last started, with no error, failed retry or dropped record. Outputs to stdout, null, counter or a file on the node do not count. A failed delivery, or an output that delivered nothing, Fails, naming the Pod and the output; where the counters cannot be read, the check is Not evaluated. Another log shipper (Vector, Fluentd, Promtail, Grafana Alloy, Filebeat, the OpenTelemetry Collector, the Datadog Agent, or Fluent Bit outside logging), recognised by its image, is named and Recorded: its configuration is not read, so where it sends logs is the organisation’s to evidence. With no log shipper, the check Fails.

Where Longhorn or Fluent Bit is not installed, the report says so. Fluent Bit’s configuration is the ConfigMap its DaemonSet in logging mounts, bke-fluent-bit on a Bahriya Kubernetes Engine cluster; a release of another name is added to fluentBit.configMaps. From Fluent Bit’s configuration the collector keeps only each output’s plugin, host and bucket; its credentials are read with it, dropped at once, and written nowhere. Fluent Bit’s counters are read directly from each Fluent Bit Pod’s HTTP server, which its chart serves on port 2020 by default (fluentBit.metricsPort). While the collector delivers by email or to S3 and its NetworkPolicy applies, that policy admits this one port, on Pods labelled app.kubernetes.io/name: fluent-bit in logging, and nothing else in that namespace. A read that is refused or times out leaves log forwarding not evaluated, with the reason; it is never passed. The collector keeps the counters and nothing else. From a backup target it keeps only the scheme of its address; from a Velero storage location, its provider and the names of its encryption settings, not its bucket or their values. It reads no Secret.

Activating a regime

Each regime is licensed on its own, and the Convoy is licensed with the first regime. There is no licence for every regime: a licence grants the regimes it names, and no others. To add a regime to a running release:

  1. Have the regime added to the licence. Ask Mamluk to add the regime to your organisation’s Bahriya Tersane licence: in the Mamluk console, open Support, choose Raise a ticket, choose “Licences and clusters” under “What is this about” and “Bahriya Tersane and its Convoys” under “Which product”, and name the regime; or ask your organisation’s contact at Mamluk. Any member of the organisation may raise a ticket. Mamluk changes the licence; the console shows it, but cannot change it.

  2. Confirm the grant in the console. On the organisation’s Tersane page in the Mamluk console, the licence names the Convoy with each regime it grants, such as “Compliance and Assurance Convoy (NCA ECC, QCB)”, and the images the licence may pull include the regime’s evidence model:

    docker pull 1x.ax/tersane/convoy/compliance/regimes/qcb/catalogue:<tag>
    

    The chart repository follows a change to the licence on its next request. The registry follows it within 15 minutes.

  3. Check the credentials. No new credential is needed: the organisation’s chart credential and image tokens serve every regime the licence grants. The pull secret named in collector.imagePullSecrets (1x-ax by default) must exist in the release’s namespace and in the node collector’s namespace (nodeCollector.namespace, by default the release’s namespace with -node added). The chart creates the node collector’s namespace, so its pull secret is created after the first install, as step 5 of Installation shows; never create that namespace yourself. Access and credentials describes the credentials and how to create the pull secret.

  4. Add the regime to the values and upgrade. The chart version must hold the regime: it is listed in the table above in that version’s README, and the chart refuses a regime it does not hold. In the values file you keep:

    regimes:
      - nca-ecc
      - qcb
    

    then:

    helm repo update
    helm upgrade <release> compliance-convoy/compliance \
      --namespace <namespace> -f <values file>
    

    The upgrade adds the regime’s CronJob, <release>-collect-qcb, and its evidence volume, <release>-evidence-qcb, and restarts the web view so that it serves the new volume.

  5. Start a first run and read its report. Rather than wait for the schedule:

    kubectl -n <namespace> create job collect-qcb-now --from=cronjob/<release>-collect-qcb
    kubectl -n <namespace> wait --for=condition=complete job/collect-qcb-now --timeout=30m
    

    The run’s report is then in the web view, under the regime, and on its volume, and it is sent by email and to object storage where those are configured. A run whose Pod does not start with an image pull error is a regime the licence does not yet grant to the registry, or a missing pull secret: check steps 2 and 3.

Removing a regime. Remove it from regimes and upgrade. The regime’s CronJob is removed and the web view no longer serves it. Its evidence volume is kept, with every run on it, so that the evidence remains available; delete it with kubectl -n <namespace> delete pvc <release>-evidence-<regime> when it is no longer needed.

When a regime is no longer licensed. Each run pulls the regime’s image, so the registry is asked on every run. A regime that the licence no longer grants stops at the pull: the Pod does not start, the collector does not run, and nothing is written. When one regime is removed from the licence, the others carry on; when the licence is suspended, revoked or expires, every regime stops. The runs already on the volumes are not touched. When the regime or the licence is restored, the next scheduled run proceeds.

Scoped assessment

By default the Convoy assesses every namespace of the cluster. An audit scope is often narrower. List the namespaces in scope, and the Convoy assesses only those:

scope:
  namespaces:
    - payments
    - payments-batch
    - kube-system

With a list:

  • the collector reads workloads, reports, role bindings, Ingresses and Velero’s objects only in the listed namespaces. Its permission to read them is a read-only Role in each listed namespace; its ClusterRole keeps only the cluster-scoped reads (nodes, namespaces, ClusterRoles and their bindings, the admission policies, cluster-scoped reports and StorageClasses). It cannot read a namespace that is not listed;
  • the admission policies and Kyverno policies apply only in the listed namespaces, and still not in an exempt one;
  • each listed namespace must exist when the chart is installed.

What the report states. The report’s first page, the PDF, the email, the OSCAL output and the Compliance App’s run screen state the scope in one sentence, for example:

Assessed: the 18 listed namespaces. Not assessed: 4 other namespaces of this cluster. This report is not evidence for them.

The namespaces that were not assessed are counted, never named. Each requirement the cluster evidences also says where its evidence comes from: the listed namespaces only, or the whole cluster, for the checks of the nodes, the platform, backups and log forwarding. A listed namespace that the cluster does not have is named.

List the whole audit scope. A namespace in your auditor’s scope that is left off the list leaves every requirement it bears on unproven. Agree the list with your auditor.

Platform evidence. Some evidence sits in a fixed namespace: the control plane’s settings in kube-system, the platform’s settings in platform.namespace (bahriya-system by default), Fluent Bit’s configuration in logging and Longhorn’s backups in longhorn-system. When that namespace is not listed, it is not read, and the checks that need it are Not evaluated. List it to have them judged. The node collector finds the etcd data directory and the API server’s audit log directory from kube-system. When kube-system is not listed, it is not read: every other node fact is gathered and judged, the etcd data directory and audit log permissions checks are Not evaluated, and each requirement evidenced by the audit log permissions check says beside its result that the audit log directory was not examined. List kube-system to have both judged.

Assessing the platform’s own namespaces. By default the admission and Kyverno policies do not apply in policies.exemptNamespaces, which holds the namespaces of a Bahriya Kubernetes Engine cluster’s own components. A scope that lists those namespaces, as an assessment of the platform itself does, therefore leaves the policies applying nowhere in it. To judge them, remove the listed namespaces from policies.exemptNamespaces (the list replaces the default as a whole) and set policies.defaultAction to Audit: Audit records each result in the policy reports and never warns or refuses a request. The platform’s components need host access and privileges by design, so expect many recorded failures there.

Scoping changes nothing commercially: the licence is the same, per regime and per organisation; scope is an assessment setting, not a tier.

Keeping the engines inside the scope

The claim that an unlisted namespace is never read is made for the Convoy’s collector. Kyverno and trivy-operator are separate charts, and by default they work in every namespace. To keep them to the same namespaces, set them when you install them:

  • trivy-operator scans and writes its reports only in the namespaces named in engine.targetNamespaces, a comma-separated list. Give it the same list as scope.namespaces:

    engine:
      targetNamespaces: "payments,payments-batch,kube-system"
    
  • Kyverno applies the Convoy’s policies only in the listed namespaces, because the Convoy’s policies select them, and writes their results only there. Its webhooks can also be limited to chosen namespaces, so that no admission request from any other namespace reaches it, and made to fail open. The kyverno page gives the values.

Both engines keep the cluster-wide read permission their charts give them.

Which reports the collector reads. The Convoy reads its policy results from Kyverno’s policy reports, PolicyReport and ClusterPolicyReport in wgpolicyk8s.io/v1alpha2, which Tersane’s kyverno chart writes, and from the openreports.io/v1alpha1 reports on a cluster that serves that API. A report type the cluster does not serve is not read. It reads trivy-operator’s reports in aquasecurity.github.io/v1alpha1. Namespaced reports are read only in the listed namespaces.

Evidence

One CronJob per regime, <release>-collect-<regime>, runs the evidence collector (daily at 02:17 by default, collector.schedule). It mounts the regime’s image as a read-only image volume, pulled on every run, so the licence is asked each time: a regime that cannot be pulled stops at the pull, the collector never starts, and the other regimes’ runs carry on. It reads the cluster with a ClusterRole that may only get and list, Velero’s backup storage locations and schedules and the cluster’s Ingresses among what it reads. Where the namespaces exist, it may also read Longhorn’s backup targets, recurring jobs and volumes in longhorn-system, and the ConfigMaps named in fluentBit.configMaps in logging, by name (longhorn.createBinding and fluentBit.createBinding create these where the chart is rendered with helm template). It is granted no Pod proxy, exec, attach or port-forward. The only thing it writes through the API is the node collector’s Jobs, in a namespace of their own (below).

Each run is a directory on the regime’s volume, <release>-evidence-<regime>, named by its UTC time: the report in HTML and PDF, OSCAL assessment results, the full results, the snapshot of what was read, and a SHA-256 manifest, signed when evidence.signingKey.existingSecret names a Secret holding a PEM private key. Runs older than evidence.retentionDays (400) are removed. The volume is kept when the regime or the chart is removed.

The report’s “Fix first” section, after the summary, lists every object that fails a check, grouped by namespace, the most severe first. For each it gives the requirements it fails, what is wrong, how to fix it, and the kubectl command that shows the full result in trivy-operator’s or Kyverno’s report. Fix text for trivy-operator’s checks is trivy-operator’s own, labelled so, except where the Convoy replaces it; the rest is the Convoy’s. The same severity, fix text and command are in results.json and in the OSCAL assessment results.

To start a run now:

kubectl -n <namespace> create job collect-now --from=cronjob/<release>-collect-nca-ecc

What each outcome means

Each requirement has one outcome, and the report, the App and the OSCAL output use the same words:

Outcome Meaning
Fail The cluster was checked and something does not meet the requirement. The findings say what and where.
Not provided A control the requirement needs is switched off on this cluster.
Incomplete Everything that could be checked passed, but part of the requirement could not be checked in this run. The reason is given.
Not evaluated Nothing behind this requirement could be checked in this run. The reason is given. It is not a pass.
Pass Every check behind the requirement passed on this run’s date. Where coverage is partial, the pass covers the cluster’s part only.
Recorded Facts collected for the organisation’s reviewer, such as an inventory or a list of access; no verdict.
Nothing to check The cluster has nothing of this kind yet, for example no Ingress to check for TLS. Neither a pass nor a failure.
Organisational evidence The organisation evidences this requirement outside the cluster; the Convoy checks nothing here.

Each organisational requirement also says what kind of evidence it asks for: technical, outside the cluster (held by IT or security engineering), non-technical (policy, people, premises, law or contracts), or both. The App filters by it: in the cluster, technical, outside the cluster and non-technical. Each NIST SP 800-53 control is named by its title, and the report lists NIST’s statement of it with a link to NIST.

On a cluster that is not a Bahriya Kubernetes Engine cluster, or one before release 2.3, the cluster declares no control-plane settings to the collector. Where its control plane runs as static Pods, as kubeadm and kind run it, the collector reads audit logging and its retention, anonymous access, encryption of Secrets at rest and the control-plane TLS settings from the API server’s own settings, and judges them. The other platform requirements, and all of them on a managed control plane such as EKS, AKS or GKE, are organisational evidence, shown under technical, outside the cluster, with the reason.

Pass, with exception

The Convoy accepts one finding that the engines report on a template which the running containers do not share. Envoy Gateway writes each Envoy proxy Deployment with a container named shutdown-manager that has requests and no limits, and gives no setting for its limits. The resource-limits policy and trivy-operator’s checks AVD-KSV-0011 and AVD-KSV-0018 judge the Deployment’s template, so they fail on it, although a LimitRange in the namespace gives the container its limits when each Pod is admitted. The sipar-gateway chart installs that LimitRange.

The finding is accepted in a run only when every one of these holds:

  • the finding names the container shutdown-manager and no other container;
  • its workload owns at least one running Pod that carries Envoy Gateway’s proxy labels (app.kubernetes.io/name=envoy, app.kubernetes.io/component=proxy, app.kubernetes.io/managed-by=envoy-gateway, and either both gateway.envoyproxy.io/owning-gateway-name and gateway.envoyproxy.io/owning-gateway-namespace, or gateway.envoyproxy.io/owning-gatewayclass);
  • every such running Pod’s shutdown-manager has a CPU and a memory limit;
  • the collector read the namespace’s LimitRanges, and one sets a Container default for both CPU and memory.

Anything missing leaves the finding a failure, with the reason beside it. A check whose only failures are accepted findings shows Pass, with exception in the report, the email, the web view and the Compliance App, never a plain Pass; the accepted finding is listed under the check with what was excepted, why, and the live limits and LimitRange read in the run. Any other failure keeps the check failed, and the accepted finding is still listed. The requirement counts as a pass, and the summary shows how many passes have such a check behind them (“of which with exception”). In the OSCAL output the observation is satisfied, carries the property exception with the value shutdown-manager-limitrange, and names the Pods, their limits and the LimitRange as its evidence.

The collector reads LimitRanges from the chart release that introduced the exception on. A cluster on an earlier chart cannot read them, so the exception is never applied there. The exception is the only one; it is part of the collector and cannot be extended through values. When Envoy Gateway gives a setting for the container’s limits and the template carries them, the checks pass without it.

The web view

The web view serves every regime’s runs from their volumes, read-only, the newest first, with each run’s report, OSCAL assessment results, results, manifest and signature:

kubectl -n <namespace> port-forward service/<release>-web-view 8080:80

then open http://localhost:8080/. It is static files served by nginx: it writes nothing, holds no credential and no service account token, and has no authentication of its own.

Who can read it. By default nothing reaches it over the network, so it is read only through kubectl port-forward, by those whose Kubernetes role allows pods/portforward in the release’s namespace. To publish it, add an Ingress of your own and admit your ingress controller in webView.allowFrom; the Ingress, and the authentication in front of it, are then yours.

A run waits for the web view’s node. Each evidence volume is ReadWriteOnce, and the web view holds them mounted, so each collector run is scheduled to the web view’s node and waits while that node is unavailable. webView.enabled=false removes the web view and the constraint.

The Compliance App

complianceApp.enabled=true serves the reports, the requirements and what to fix first behind your identity provider or local accounts, with an access log, on the same evidence volumes. It takes the web view’s place. It reads the evidence read-only, holds no service account token, and needs nothing from the Kubernetes API. Install the chart once on a cluster: its policies have fixed names, and Helm refuses a second installation.

complianceApp:
  enabled: true
  oidc:
    issuer: https://idp.example.com/realms/example
    clientId: compliance
    audience: compliance
    jwksUri: https://idp.example.com/realms/example/protocol/openid-connect/certs
    egressCidrs: ["192.0.2.10/32"]
  roles:
    - {group: platform-admins, role: administrator}
    - {group: auditors, role: auditor}
    - {group: team-a, role: namespace-owner, namespaces: [team-a]}
  ingressFrom: [ingress-nginx]

With the identity provider, the chart refuses to render without the issuer, the client, the audience, the address of its keys, its network addresses and an administrator mapping. The address of its keys, oidc.jwksUri, is the jwks_uri in its discovery document, at https://<issuer>/.well-known/openid-configuration. Publish the Service compliance-app, port 80, with an Ingress of your own and your certificate.

Signing in. complianceApp.auth.mode is oidc (the default), through your identity provider; local, with local accounts; or both, each person through one of the two. See Local accounts.

What the app may reach. With the identity provider, a NetworkPolicy limits it to DNS and to oidc.egressCidrs on port 443. The value is required and has no default. ["0.0.0.0/0"], for a provider whose addresses change, means the app may open connections to any address on that port: that nothing leaves the cluster then rests on the app’s code and not on the network.

With local accounts alone, the NetworkPolicy allows no egress at all, not even DNS.

Through a proxy. Where a proxy is the only way out of the network, set oidc.httpsProxy to its address, http://host:port. The app’s one outbound request, for the provider’s keys, then goes through the proxy, which tunnels it to the provider with CONNECT: TLS runs from the app to the provider, and the provider’s certificate is checked as before. oidc.egressCidrs and oidc.egressPort then name the proxy’s addresses and port, so the app reaches nothing but the proxy, and the proxy decides where traffic goes. People’s browsers still reach the provider for sign-in over their own network.

complianceApp:
  oidc:
    httpsProxy: http://proxy.example.com:3128
    egressCidrs: ["192.0.2.20/32"]
    egressPort: 3128
    proxyCredentialsSecret: compliance-app-proxy

If the proxy needs a username and a password, create a Secret with them under the keys username and password, and name it in oidc.proxyCredentialsSecret. They are sent to the proxy alone and never logged. oidc.noProxy lists hosts fetched directly despite the proxy, separated by commas. The app reads no other proxy setting, so a proxy configured elsewhere in the cluster’s environment is not used. The chart refuses an https:// or socks5:// proxy, an address that holds credentials, and a credentials Secret without a proxy.

Local accounts

Local accounts sign people in with a username and a password, checked against hashes held in a Kubernetes Secret. They are for evaluation, and for use before your identity provider is connected. What they lack:

  • no multi-factor authentication: a password alone signs a person in;
  • no central offboarding: a person who leaves keeps their account until it is removed from the Secret; disabling them in your directory does nothing here;
  • no password reset, expiry or self-service: the administrator changes the Secret.

The app’s settings and role-mapping screens state the same. With both, local accounts sit beside your identity provider: they bypass it, its multi-factor authentication and its offboarding, and they still sign in when it cannot be reached. Each local account appears with the prefix local: in the access log and on every screen, and its role comes from the Secret alone, never combined with a group mapping.

The Secret has one key, users.yaml. Each account has a hash, one role, for namespace-owner its namespaces, and an optional expiresAt (a date is the last day it signs in). It must hold at least one administrator.

users:
  a.rahman:
    hash: "$2y$12$..."
    role: administrator
  payments-team:
    hash: "$2y$12$..."
    role: namespace-owner
    namespaces: [payments]

Each hash is bcrypt of cost 12 or more, or argon2id. A plaintext password, or any record the app cannot accept, stops the app at start with a message naming the account and the reason, never the value. Make a hash with the app’s own command, which asks for the password twice without showing it and prints the hash alone:

kubectl run hash-password --rm -it --restart=Never \
  --image=<the app's image, as the chart pins it> \
  --command -- /var/www/bin/hash-password

or with htpasswd -nbBC 12 <username> <password>, taking the part after the colon (that puts the password on a command line). Write the file, then create the Secret from it, so that no hash is typed on a command line:

kubectl -n compliance create secret generic compliance-local-accounts --from-file=users.yaml
complianceApp:
  enabled: true
  auth:
    mode: local
    local: {existingSecret: compliance-local-accounts}
  ingressFrom: [ingress-nginx]

A change to the Secret needs no restart. Kubernetes updates the mounted file within its sync period, and the app takes it up at the next request after that: a removed account is refused, and its sessions end; a changed role is in force. On a kind test cluster a removed account was refused 68 to 85 seconds after the Secret was changed, in three runs. kubectl rollout restart deployment/compliance-app makes it sooner. A change the app cannot accept is refused, the accounts already in force stay, and the refusal is in the Pod’s log.

Sessions. A correct password gives a 15-minute access token, signed by the app, which the browser renews while the session is active and holds in memory only; the renewal uses a cookie the page cannot read. The session ends after the settings screen’s limit without activity, at sign-out, and when the account leaves the Secret. The key that signs these tokens is generated on the app’s volume at first start, or is the key signing-key.pem in the Secret complianceApp.auth.signingKeySecret names. Replacing it ends every local session. The issuer of these tokens, complianceApp.auth.local.issuer, must differ from your identity provider’s.

Failed sign-ins. Every attempt is in the access log, with the username tried, the address and the reason; the password never is. Any failure gets the same answer. After 5 failures within 15 minutes, a username, and an address, must wait 30 seconds, doubling with each further failure to at most 15 minutes.

Memory. A report too large for phpMemoryLimit is listed as unreadable, with its files still available. To read it, raise phpMemoryLimit and resources.limits.memory together and restart. The chart refuses a container limit below (workers + 1) x phpMemoryLimit + 256Mi, under which a large report would have the container killed and would not fail with an error.

The access log’s key. Each entry is signed with a key held in a Secret. Helm generates it at installation and keeps it, across upgrades and after the chart is removed. Where the chart is rendered with helm template, as GitOps tools do, you must set complianceApp.accessLogKeySecret. The chart cannot read the cluster there, so without it each render generates a new key and the tool applies it. The chart cannot tell such a render from a first installation, and does not refuse it. Create the Secret first, and keep it outside the rendered manifests:

kubectl -n <namespace> create secret generic compliance-access-log-key \
  --from-literal=access-log-key="$(openssl rand -hex 32)"

To keep a key Helm has generated, name its Secret, <release>-app-signing.

If the key changes, the entries already written are not lost, and they are not passed as good: the app has no other copy of the old key, so each is marked “Does not match its signed record”, with a notice above the access log that names both causes, an edit or a changed key. The mark itself does not tell them apart. Restoring the old key and restarting the app clears the mark on those entries, and it then falls on what the other key signed. No alarm is raised. After changing the Secret’s contents, restart the app, which reads the key when it starts.

What the signature shows, and what it does not. Each entry is signed on its own. An entry edited in the database shows; an entry deleted from it does not, and leaves no trace in the others. The Pod’s log holds the same records, so collecting it in your own log system gives you a copy to compare. The app’s maintenance checks the newest 200 entries every 5 minutes, and a mismatch appears on the administration pages and as an error line in the Pod’s log.

A run waits for the app’s node, as it does for the web view’s.

Monitoring. Port 9100 serves /metrics, with no token. It is not on the Service, and the NetworkPolicy admits only the peers in complianceApp.metrics.allowFrom, none by default. A Netdata collector on the same node reaches it as the node, which a NetworkPolicy does not stop. The Pod carries the prometheus.io scrape annotations. The name of the Deployment, compliance-app, and of its container, web, are fixed because Netdata’s alarms name them.

Alarms on a Bahriya Kubernetes Engine cluster. Current BKE releases give Netdata five alarms on the app; BKE’s release notes name the release that added them, and earlier releases have none of them. The alarms are evaluated on the cluster’s Netdata parent. compliance_app_metrics_writes raises when the app stops writing its metrics, and compliance_app_metrics_scrape when Netdata, while it is itself running and connected, cannot read them. compliance_app_run_stale raises when a regime has no run as recent as collector.schedule and collector.activeDeadlineSeconds allow: 24.5 hours on the default daily schedule, or complianceApp.runStaleAfterHours. A suspended CronJob therefore raises it. compliance_app_storage_level warns as a volume fills and is critical near full. compliance_app_not_ready raises when the Deployment has had no ready replica for three minutes.

  • Delivery is off until you turn it on. Netdata shows the alarms, but they reach a person only once a notification channel is turned on in BKE’s Netdata health settings, where BKE ships every channel off, or the cluster’s Netdata is connected to Netdata Cloud with notifications set there. Once a channel is on, Netdata’s own stock alarms are sent on it as well.
  • A run that fails before it can record why appears only through compliance_app_run_stale, so up to 24.5 hours later on the daily schedule. This happens, for example, when the collector cannot start or is killed for memory. A run that fails after it starts records why, and that appears on the Settings screen when the run ends.
  • What raises no alarm: the Deployment deleted; Netdata’s k8s-state collector or its parent down; and a Ready Pod without the scrape annotations. Each of these is silent, so check them when you change the cluster or Netdata.
  • While Netdata’s agent on a node is disconnected from its parent, the parent shows none of that node’s alarms, these included, and in testing they had not reported again several minutes after the agent reconnected. Do not rely on these alarms alone to tell you that the app’s metrics cannot be collected.
  • The alarms rely on Netdata’s collector on the same node reaching the Pod as that node, as described under Monitoring above.

When the evidence is overdue. The app and the alarm compliance_app_run_stale use one rule. A regime’s newest run is overdue once more time has passed since it started than collector.schedule and collector.activeDeadlineSeconds allow, 24.5 hours on the default daily schedule, or complianceApp.runStaleAfterHours. The app then shows the auditor and the lead a notice against that regime on the Regimes list and on its newest run: no run has completed since that run, by when one was expected, and, where the last attempt failed, when it failed. The reason for a failure is shown only to administrators, on the Settings screen. Times are shown in UTC. A schedule that cannot be read is judged on a daily gap, and the Settings screen says that it cannot be read.

With no run yet, the time runs from the app’s first start, which the app records once on its own volume, <release>-app-data. If that record cannot be written or read, a regime with no run is shown as overdue, and the reason is in the Pod’s log. The record is published with a hard link, so that two processes starting together record one time: the volume’s filesystem must support hard links, as Longhorn’s volumes and local volumes do.

When a process stops. Beside the web server, the container runs a process that writes the metrics and one that maintains the database. Each is started again if it stops. If one hangs, or cannot be started again, the alarm compliance_app_metrics_writes raises, and the container is restarted some minutes later: for the metrics, the alarm after about 2 minutes and the restart after about 6; for the maintenance, the alarm about an hour after its last pass and the restart at about 66 minutes. You see the alarm raised and then cleared, an event Liveness probe failed on the Pod that names the process, and a higher restart count. The access log and the run index are on the app’s volume, and the evidence is not changed. Where the alarm returns after each restart, read the Pod’s log and the Settings screen.

Organisational evidence in the Compliance App

The cluster cannot evidence a requirement about a policy, a committee or a contract. For each organisational or partly covered requirement, the role evidence-owner adds the organisation’s own evidence in the app: comments, attached files, a named owner and a review date. auditor and lead read it; namespace-owner and administrator do not. It is shown as provided by the organisation, apart from the outcome, and never changes a verdict, a count or a file of the collector’s. It carries from run to run.

complianceApp:
  roles:
    - group: compliance-evidence-owners
      role: evidence-owner
  evidence:
    clamav:
      host: clamav.clamav.svc.cluster.local
      namespace: clamav

Files are scanned before they are kept. The app needs a clamd it can reach over clamd’s INSTREAM command. The supported installation is Tersane’s ClamAV chart at its current version, beside the Convoy: name its Service in complianceApp.evidence.clamav (host, port and namespace). That chart runs ClamAV as a non-root user with CPU and memory limits, so it runs in a namespace that enforces the restricted Pod Security Standard and in one whose quota requires limits. Each file is scanned by clamd before it is accepted; an infected file, or a scan that cannot be made, is refused, nothing is stored, and the access log records the reason. Each accepted file records the version and date of the signatures it was scanned with, on the item, in the access log and in the export. While those signatures are older than complianceApp.evidence.clamav.maxSignatureAgeDays (7 days by default), files are refused, and the refusal gives their date: signatures that do not update usually mean freshclam cannot reach its updates, so check the ClamAV chart’s proxy and private mirror settings. complianceApp.evidence.maxFileMiB must not exceed the largest stream the scanner accepts (clamd’s StreamMaxLength); a larger file is refused by the scanner. Without a scanner, or where it cannot be reached, the app says so on the evidence screen and in each refusal, names the chart and the values to set, and comments, the owner and the review date can still be recorded. With one, the app’s NetworkPolicy admits DNS and the ClamAV Pods on clamd’s port, in that namespace, and nothing else of the cluster, with local accounts as with an identity provider.

What can be attached. PDF, PNG, JPEG, plain text, CSV, and Word, Excel and PowerPoint documents, judged by their content as well as their name, up to complianceApp.evidence.maxFileMiB (25 MiB) each. No archives, and no document with macros. A file is always downloaded as an attachment, never shown in the browser. Files are kept on their own volume, <release>-app-files (complianceApp.evidence.storage, 5 GiB), which stays when the chart is removed.

Removal and reviews. A removed item stays listed, marked removed, with who removed it and when; its file is deleted once it has been removed for longer than the access log’s retention (400 days by default). A review date that has passed is flagged on the requirement and counted on the regime. The app sends no email.

The export. An auditor or a lead downloads a run with the organisation’s evidence as one ZIP file: the run’s files unchanged, with the collector’s manifest and signature; the organisation’s items, owners and files, with an OSCAL assessment-results file of their own whose observations are marked as provided by the organisation; and export-manifest.sha256, the digest of every file, signed with the app’s own ECDSA P-256 key. The settings screen shows the public key. Check a package with:

cosign verify-blob --key export.pub --signature export-manifest.sha256.sig \
  --insecure-ignore-tlog export-manifest.sha256
sha256sum -c export-manifest.sha256

Helm generates the key at installation and keeps it. Where the chart is rendered with helm template, as GitOps tools do, set complianceApp.evidence.exportKeySecret to a Secret of yours whose key cosign.key is an unencrypted ECDSA P-256 private key in PEM; without it, each render would generate a new key, and exports already given to an assessor would no longer verify against the key the app shows.

Email

With email.mode: smtp, each run is sent through your own SMTP server: the summary and “Fix first” as the message, with the PDF report attached, and the HTML report too if email.attach names it, up to email.maxAttachmentMB.

email:
  mode: smtp
  smtp:
    host: smtp.example.com
    port: 587
    tls: starttls          # or implicit
    existingSecret: smtp   # keys username and password
    egressCIDRs: [192.0.2.25/32]
  from: evidence@example.com
  to: [security@example.com]

TLS is required: with starttls a server that does not offer STARTTLS is refused, and the server’s certificate is always verified, against the system’s CAs or the CA in email.smtp.caSecret. Nothing is sent while email.mode is none, the default.

Object storage

With s3.enabled, each run’s files are uploaded to your bucket as <prefix><regime>/<run>/<file>, over HTTPS with the store’s certificate verified. s3.endpoint is required while S3 delivery is on, as https://<host>[:<port>]: an S3-compatible store, such as MinIO or Ceph, or, for Amazon S3, https://s3.<region>.amazonaws.com. The chart refuses to install without it and does not choose a provider for you. s3.pathStyle puts the bucket in the path, as many S3-compatible stores expect. The keys are in the Secret s3.existingSecret (accessKeyId, secretAccessKey).

Evidence stays on the cluster’s volume unless you configure delivery by S3 or by email. An endpoint, mailbox or mail relay outside the country whose rules apply places the evidence outside it.

What delivery records, and the collector’s traffic

Each delivery is recorded in the run as delivery.json. It is written after the run’s manifest is signed, so it is not covered by the manifest. A delivery that fails is recorded and logged, and the run still completes: the evidence is on the volume whether or not it was sent.

While email or object storage is on, a NetworkPolicy limits the collector’s traffic to DNS, the Kubernetes API server, Fluent Bit’s HTTP server on fluentBit.metricsPort (2020) on Pods labelled app.kubernetes.io/name: fluent-bit in logging, and the addresses you give in email.smtp.egressCIDRs and s3.egressCIDRs; the chart refuses to install without them. If your Fluent Bit serves its HTTP server on another port, set fluentBit.metricsPort to it, or log forwarding is not evaluated. To allow any address, write 0.0.0.0/0 and ::/0.

After adding a control-plane node, upgrade the chart. The API server’s addresses are read from the kubernetes EndpointSlices in default when the chart is installed or upgraded. Until the next upgrade, the collector cannot reach the new node’s API server. Where the chart cannot read them, for example when a GitOps tool renders it, give them in apiServer.cidrs.

Storage that binds on first use. Where the storage class binds a volume only when a Pod first uses it (volumeBindingMode: WaitForFirstConsumer), the evidence volumes stay Pending until each regime’s first run, and helm install --wait waits for them until it times out. Install without --wait, or start a first run as above.

On a Bahriya Kubernetes Engine cluster at release 2.3 or later, the collector reads the platform’s facts from the ConfigMap bke-platform through BKE’s ClusterRole bke-platform-reader, bound in platform.namespace when the chart is installed or upgraded there. Elsewhere the platform checks are reported as not evaluated, never as failed.

Node facts

What the API cannot show, the kubelet’s settings, the modes of the files Kubernetes keeps on each node and the node’s time service, is read on the node itself. At each run the collector starts one short Job per node in the namespace nodeCollector.namespace (the release’s namespace with -node added, by default). Each Pod:

  • runs as user 65534, with every capability dropped and no service account token;
  • mounts only named host paths, each typed as a file or a directory and read-only, so a path that is absent is never created on the node;
  • reads the kubelet’s configuration file and its flag files, and records the mode and owner of the files it cannot open;
  • prints the node’s facts to its log, where the collector reads them, and sends nothing over the network.

The host paths mounted, each read-only, are /etc/kubernetes, the kubelet’s configuration, flag file and certificates, /etc/default and /etc/systemd; on a control-plane node, the etcd data directory and the API server’s audit log directory, each taken from the API server’s or etcd’s own volume.

The audit log directory. It is judged by its own check, the audit log permissions check, and not by the node file permissions check. The audit log check is judged when the API server writes its audit log to a directory on each control-plane node, held in a host-path volume; on a Bahriya Kubernetes Engine cluster, from the release that sets these modes, which the check’s reason names.

The node collector mounts the directory read-only and examines modes and owners only; no code in it opens a file there. Because it runs as user 65534, it can read the audit log’s content only where the directory and a file are already open to other users, which is the fault the check reports. On a correctly set node it cannot list the files: a directory owned by root and no more open than 700 then decides for them, since no user but root can reach them, whatever their own modes. The check fails a wider directory, a file wider than 600 or an owner other than root, and names the remedy: chmod 0700 the directory, chmod 0600 each file in it and chown root:root, as root on the node named. On BKE, upgrade.sh with --commit sets them; a node moved to such a release by apply.sh alone keeps its earlier modes until then.

The check passes only when the directory was examined on every control-plane node that runs an API server. Otherwise it is Not evaluated, and says why, beside each requirement that rests on it:

  • the API server writes no audit log file, or writes it to its standard output;
  • the audit log path is not absolute, or lies in no volume, or in a volume that is not a host directory;
  • no node runs an API server Pod, as with a managed control plane;
  • kube-system is not in the run’s scope;
  • the cluster runs a Bahriya Kubernetes Engine release that does not set these modes: the reason names the release to upgrade to.

The namespace is labelled Pod Security privileged, because every other level refuses host paths. It holds nothing but these Jobs, the Convoy’s own policies do not apply in it, and a NetworkPolicy allows no traffic in or out. The collector may create, read and delete Jobs there and read their Pods and logs; it cannot create a Pod directly. Facts read within nodeCollector.maxAgeMinutes (60) by another regime’s run are used again.

The collector’s pull secrets must also exist in this namespace, since the node Jobs run the collector’s image.

The kubelet’s settings are those of its files: a change made after the kubelet last started, and not yet in effect, is reported as though it were. A node whose Job does not finish within nodeCollector.timeoutSeconds is reported as not read, with the reason, and its checks as not evaluated. nodeCollector.enabled=false removes the namespace and leaves every node check not evaluated.

To pull the images from a mirror of your own, set collector.image.repository and regimeImages.repositoryPrefix; each image is still pulled by the digest the chart pins, so copy with skopeo copy --all --preserve-digests.

In-cluster summary

The chart also creates trivy-operator’s ClusterComplianceReport tersane-compliance-nist: trivy-operator’s checks grouped by the NIST SP 800-53 Rev. 5 controls the Convoy maps them to, refreshed every six hours (summary.cron), for a quick view in the cluster:

kubectl get clustercompliancereport tersane-compliance-nist -o wide

It is indicative, not evidence. trivy-operator shows a check that has not run as passed, and the summary covers only trivy-operator’s checks, keyed on NIST rather than on any regime. The evidence for a regime is the report the collector writes, which computes every result from the raw reports and records a check with no result as not evaluated, and says what to fix. It is the same whichever regimes the licence grants. summary.enabled=false removes it.

Checks the Convoy does not use, and why

The report names, for each of the Convoy’s checks, the trivy-operator checks it is evidenced from. Where a check of trivy-operator’s would give a wrong result for the clusters the Convoy serves, the Convoy does not use it, and the check is listed here with the reason.

AVD-KSV-0125: images from trusted registries

AVD-KSV-0125 reports an image as coming from an untrusted registry unless its registry ends with an entry of a built-in list. That list names the registries of three providers only: Microsoft Azure (azurecr.io), Amazon Elastic Container Registry (the ecr.<region>.amazonaws.com domains) and Google (gcr.io, us.gcr.io, eu.gcr.io and asia.gcr.io). All three providers are based in the United States. Every other registry is reported as untrusted: a registry in your own data centre, a registry in a national or in-country cloud, and 1x.ax, from which Tersane’s own images are served.

For an organisation whose regulator or policy requires its systems and data to remain in the country, or inside its own perimeter, the result therefore reflects whether an image comes from one of those three providers, not whether it comes from a source the organisation trusts. Every registry the organisation runs itself, and every in-country registry other than those three providers’, is reported as untrusted.

The list cannot be replaced without losing the other checks. The check reads a replacement list as rule data, which the trivy-operator version the trivy-operator chart installs accepts only as a custom rule or library in its policies ConfigMap; with any custom rule or library present, it reports no built-in configuration check at all. Measured on a test cluster, each workload’s report fell from 55 checks to none, with no error logged.

The Convoy judges the same property against the registries you name instead. You list your authorised registries in registries.allowed, each as the beginning of an image reference (1x.ax/ by default). policy.registries-allowed warns about an image from any other registry, or refuses it once policies.actions.registries-allowed is Deny, and the report evidences the result for every workload. trivy.image-sources continues to use AVD-KSV-0034 (images from public registries) and AVD-KSV-0013 (images referenced by a moving tag).

Admission policies

Eleven ValidatingAdmissionPolicy objects, built into Kubernetes, so they work whatever policy engine the cluster runs, or none. Each judges an object as it is created or changed; Kyverno reports each one’s results for the objects already in the cluster.

Policy Applies to Requires
registries-allowed Workloads Images only from registries.allowed
no-latest-tag Workloads Every image by a fixed tag or digest
no-privileged Workloads No privileged container; no capability added but NET_BIND_SERVICE
no-host-namespaces Workloads No host network, process or IPC namespace
no-host-ports Workloads No host port
no-host-path Workloads No host path volume
no-default-token Workloads automountServiceAccountToken stated, true or false
resource-limits Workloads CPU and memory limits on every container
namespace-environment Namespaces The label tersane.bahriya.cloud/environment, one of namespaces.environments
namespace-classification Namespaces The label tersane.bahriya.cloud/data-classification, one of namespaces.classifications
ingress-tls Ingresses TLS for every host

A workload is judged at its top-level object: a Deployment, StatefulSet, DaemonSet, Job, CronJob, or a Pod or ReplicaSet that nothing owns. The ReplicaSets and Pods a controller creates are not judged again, so a warning reaches whoever applied the workload and the report holds one result for it.

Each policy is Warn by default: the object is admitted, the person who applied it sees a warning naming what failed, and the result is recorded. Audit records the result without a warning. Move a policy to Deny once nothing that should be admitted fails it, or to Off to remove it:

policies:
  defaultAction: Warn
  actions:
    resource-limits: Deny
    no-default-token: Off

No policy applies in the namespaces of policies.exemptNamespaces, nor in the Convoy’s own. The defaults are the namespaces of a Bahriya Kubernetes Engine cluster’s own components, among them netdata, and those of Kyverno and trivy-operator; their components need what the policies forbid. The report lists their failures under each check without counting them. The namespace default is judged only while it holds a workload.

Kyverno policies

What the API server cannot decide itself is done with Kyverno.

Policy Kind Establishes
images-signed ImageValidatingPolicy Each container’s image is signed with cosign by the key that covers it; the collector’s report fails an image no key covers
namespace-network-policy ValidatingPolicy Each application namespace holds at least one NetworkPolicy
default-deny GeneratingPolicy A default-deny NetworkPolicy in each application namespace, when turned on

Image signatures

Mamluk’s published signing keys are held by the chart, in files/signing/, and verify every image matching signatures.mamluk.images (1x.ax/*). Each entry of signatures.keys is a policy of its own, tersane-compliance-images-signed-<name>, for the images matching its patterns:

signatures:
  keys:
    - name: registry
      images: ["registry.example.com/*"]
      publicKeys:
        - |
          -----BEGIN PUBLIC KEY-----
          ...
          -----END PUBLIC KEY-----

An image passes a policy when it is signed with any one of that policy’s keys, so a key and its successor are listed together while a key is replaced. An image matching no policy’s patterns is not checked for a signature; registries-allowed decides where images may come from. Signatures in cosign’s bundle format and in its earlier .sig format are both verified. The policy changes nothing in the Pod: tags are not replaced with digests.

It follows policies.actions.images-signed like the built-in policies: Warn admits the Pod and warns, naming each unsigned image, Deny refuses it, and Off removes the policies.

Images in a private registry

Kyverno reads each signature from the registry that holds the image, with its own pull secrets: those named in the kyverno chart’s existingImagePullSecrets, 1x-ax by default. For an image in any other registry that refuses anonymous reads, Kyverno cannot read the signature, and the report lists the image as failing with a detail such as:

evaluation error: failed to evaluate policy: GET https://registry.example.com/v2/payments/app/manifests/<tag>: UNAUTHORIZED: authentication required

Create a Secret of type kubernetes.io/dockerconfigjson for that registry in Kyverno’s namespace, and name it in signatures.credentialSecrets. Write the credential file first, as Creating the pull secret does for 1x.ax, with your registry’s host and username in place of registry.example.com and <username>; the shell asks for the password and does not show it:

(
  umask 077
  printf 'Registry password: '; stty -echo; IFS= read -r TOKEN; stty echo; echo
  printf '{"auths":{"registry.example.com":{"auth":"%s"}}}\n' \
    "$(printf '%s:%s' '<username>' "$TOKEN" | base64 | tr -d '\n')" > registry.json
)
kubectl --namespace kyverno create secret generic example-registry \
  --type=kubernetes.io/dockerconfigjson \
  --from-file=.dockerconfigjson=registry.json
rm registry.json
signatures:
  credentialSecrets: [example-registry]

Kyverno then reads signatures with these Secrets and with signatures.kyvernoPullSecrets, and with nothing else: a policy’s credentials replace Kyverno’s own rather than adding to them, so the chart lists Kyverno’s pull secrets beside yours. Keep signatures.kyvernoPullSecrets the same as the kyverno chart’s existingImagePullSecrets. A named Secret that does not exist is skipped.

Where Mamluk’s images are mirrored into your own registry, point Mamluk’s key at the mirror, name the mirror’s credential, and leave Kyverno’s pull secrets out if no 1x-ax Secret exists:

signatures:
  mamluk:
    images: ["registry.example.com/mirror/*"]
  credentialSecrets: [example-registry]
  kyvernoPullSecrets: []

When Mamluk’s key is replaced, the new key is added to files/signing/ beside the old one, from bke/signing/, and the old one removed once every published image carries the new signature.

NetworkPolicies

namespace-network-policy is evaluated by Kyverno’s background scan only, because a namespace holds no NetworkPolicy at the moment it is created. Its results are in Kyverno’s cluster policy reports, renewed when a namespace changes and at each background scan. It is never refused at admission: policies.defaultAction does not apply to it, and Deny named for it is refused.

Namespaces in a service mesh. A service mesh pools connections, and a pooled connection opened before a NetworkPolicy change can outlive it. So the collector does not judge a meshed namespace’s segmentation from its NetworkPolicies where its traffic can be pooled; the check is then Not evaluated, and the report names each such namespace and why:

  • a namespace in Istio ambient mode, whatever its Services declare;
  • a namespace with Kuma sidecars where any Service port resolves, in Kuma’s order, to http, http2 or grpc: its appProtocol where Kuma recognises it, otherwise the <port>.service.kuma.io/protocol annotation, otherwise TCP;
  • a namespace with Istio sidecars where any Service port does not declare TCP explicitly (appProtocol: tcp, or a name beginning tcp-), because Istio detects an undeclared port’s protocol itself;
  • a meshed namespace whose Services could not be read.

A namespace is meshed when a Pod in it carries a sidecar (kuma-sidecar, istio-proxy), when a Pod’s own label or annotation enrols it, or when the namespace’s label enrols it and not every Pod opts out. A label counts by its value: for Kuma, enabled, true and yes enrol, and disabled, false and no do not; a Pod’s own value overrides its namespace’s. Segmentation in such a namespace rests on the mesh’s own traffic permissions, which the collector does not read. A failure in another namespace still fails the check. To read each Service’s port protocols, the collector may get and list Services; of each it keeps only the ports’ names, their appProtocol and the Kuma protocol annotation.

generate.defaultDenyNetworkPolicy.enabled has Kyverno write the NetworkPolicy tersane-default-deny into each application namespace as it is created. It selects every Pod and allows no traffic, in or out, except DNS lookups to the cluster’s DNS service while allowDNS is true, so each workload then needs NetworkPolicies of its own allowing what it uses. Kyverno restores it when it is deleted or changed.

What a workload loses when it is written. On a cluster whose network enforces egress, as every Bahriya Kubernetes Engine cluster’s does, every Pod in the namespace at once stops reaching:

  • other Pods and Services, in its namespace and elsewhere;
  • the Kubernetes API server, so an operator, a controller or any workload that calls the API stops working;
  • anything outside the cluster, including registries, databases and APIs it calls by address;

and nothing reaches it, ingress controllers included. Only DNS lookups remain, while allowDNS is true. Before turning it on, give each application namespace NetworkPolicies that allow its workloads’ traffic: for the API server, an egress rule to the control-plane nodes’ addresses on port 6443. existingNamespaces also writes it into the namespaces that exist when it is turned on, which stops their traffic at once. It is off by default. Turning it off again removes the policy and every NetworkPolicy it wrote.

On a Bahriya Kubernetes Engine cluster with the compliance setting. Where BKE’s compliance.convoy setting is on, install the Convoy with this profile:

generate:
  defaultDenyNetworkPolicy:
    enabled: true
    allowDNS: true
    existingNamespaces: false

Every namespace created afterwards starts closed, so its workloads need NetworkPolicies of their own from the start, and the traffic of the namespaces that already exist is left as it is. A NetworkPolicy has no mode that only warns: it blocks traffic or does nothing, so the profile enforces from the first new namespace, and a namespace that existed before it, and has no NetworkPolicy of its own, keeps failing the network-policy check until its owners add one. On a new cluster there is no such namespace. Only Kubernetes NetworkPolicies count: a Calico staged policy, which records what it would do and enforces nothing, never passes the check.

The chart adds the permissions Kyverno’s controllers need for these, through ClusterRoles that Kyverno’s own roles aggregate.

Parameters

Parameter Description Default
regimes Regimes evidenced [nca-ecc]
scope.namespaces The namespaces to assess; empty assesses the whole cluster. See Scoped assessment []
policies.defaultAction Warn, Audit or Deny for every policy Warn
policies.actions Warn, Audit, Deny or Off for a policy, by name, including images-signed and namespace-network-policy {}
policies.exemptNamespaces Namespaces no policy applies to the platform’s, Kyverno’s and trivy-operator’s
registries.allowed Registries images may come from, as reference prefixes [1x.ax/]
signatures.mamluk.images Images Mamluk’s keys verify, as patterns [1x.ax/*]
signatures.keys Keys of your own: name, images, publicKeys []
signatures.credentialSecrets Secrets in Kyverno’s namespace for reading signatures from other registries []
signatures.kyvernoPullSecrets Kyverno’s own pull secrets, listed with credentialSecrets so that they are kept [1x-ax]
generate.defaultDenyNetworkPolicy.enabled Write a default-deny NetworkPolicy into each application namespace false
generate.defaultDenyNetworkPolicy.allowDNS Allow DNS lookups in it true
generate.defaultDenyNetworkPolicy.existingNamespaces Also write it into existing namespaces false
namespaces.environments Values of the environment label [production, staging, test, development]
namespaces.classifications Values of the data classification label [top-secret, secret, restricted, public]
summary.enabled Create the NIST-keyed in-cluster summary (indicative) true
summary.cron When trivy-operator refreshes it 0 */6 * * *
collector.image.repository, .tag, .digest The collector image, pinned by digest 1x.ax/tersane/convoy/compliance/core/collector
collector.imagePullSecrets Pull secrets for the collector and regime images [{name: 1x-ax}]
collector.schedule When each regime’s collector runs 17 2 * * *
collector.activeDeadlineSeconds A run taking longer is stopped 1800
collector.scanMaxAgeHours Scans older than this fail scan freshness 48
collector.organisation, collector.cluster Shown on the report’s first page ""
collector.resources The collector’s requests and limits 100m/256Mi, 1/2Gi
webView.enabled Serve the runs, read-only, in the cluster true
webView.allowFrom NetworkPolicy peers that may reach the web view [] (port-forward only)
webView.url The web view’s address, shown in each email ""
email.mode none or smtp none
email.smtp.host, .port, .tls Your SMTP server; starttls or implicit "", 587, starttls
email.smtp.existingSecret, .usernameKey, .passwordKey The SMTP credential "", username, password
email.smtp.caSecret, .caKey A CA for the server’s certificate "", ca.crt
email.smtp.egressCIDRs The server’s addresses; required []
email.from, email.to Sender and recipients "", []
email.attach, email.maxAttachmentMB report.pdf, report.html; the size limit [report.pdf], 10
s3.enabled Upload each run false
s3.endpoint, .region, .bucket, .prefix, .pathStyle The store and where the objects go; the endpoint is required while S3 delivery is on "", "", "", "", false
s3.existingSecret, .accessKeyIdKey, .secretAccessKeyKey The access key "", accessKeyId, secretAccessKey
s3.caSecret, .caKey A CA for the store’s certificate "", ca.crt
s3.egressCIDRs The store’s addresses; required []
apiServer.cidrs, .port The API server’s addresses, where the chart cannot read them [], 6443
nodeCollector.enabled Read each node’s facts true
nodeCollector.namespace The node Jobs’ namespace, labelled privileged "" (<namespace>-node)
nodeCollector.maxAgeMinutes Node facts this recent are used again 60
nodeCollector.timeoutSeconds How long a run waits for the nodes 300
evidence.retentionDays Days of runs kept 400
evidence.size, evidence.storageClass Each regime’s volume, sized by the table in values.yaml 10Gi, the default class
evidence.signingKey.existingSecret, .key A Secret with the PEM key that signs each manifest "", key.pem
regimeImages.repositoryPrefix A mirror holding the regime images "" (1x.ax)
platform.namespace Where BKE records its platform facts bahriya-system
platform.createBinding Bind the collector there whether or not the chart can see the namespace; for helm template on BKE false
longhorn.createBinding Let the collector read Longhorn’s backup targets, recurring jobs and volumes in longhorn-system whether or not the chart can see the namespace; for helm template false
fluentBit.createBinding Let the collector read the ConfigMaps in fluentBit.configMaps in logging whether or not the chart can see the namespace; for helm template false
fluentBit.configMaps The ConfigMaps in logging the collector may read: the one Fluent Bit’s DaemonSet mounts, named after its Helm release [bke-fluent-bit, fluent-bit]
fluentBit.metricsPort The port of Fluent Bit’s HTTP server, from which the collector reads each Pod’s delivery counters; the collector’s NetworkPolicy admits this port alone, on Fluent Bit’s Pods in logging 2020
prerequisites.check Refuse to install without Kyverno and trivy-operator true