Documentation / compliance
Compliance and Assurance Convoy
Evidences a regulatory regime’s technical controls in a Kubernetes cluster.
No regime’s content is in the chart. Each regime’s evidence model is an image
of its own, 1x.ax/tersane/convoy/compliance/regimes/<regime>/catalogue, and
the chart holds only its reference, pinned by digest, in files/regimes.json.
The chart is granted with the Convoy and each regime is granted on its own, so
a regime the licence does not grant cannot be pulled, and stops alone.
Activating a regime describes how a regime is added.
Prerequisites
Kubernetes 1.36 or later, which the Bahriya Kubernetes Engine provides from release 2.2: each regime’s evidence model is mounted as an image volume, and the container runtime must support image volumes.
The Bahriya Tersane kyverno and trivy-operator charts, installed
separately. The chart refuses to install without them. To render it outside a
cluster, pass their APIs:
helm template compliance . \
--api-versions policies.kyverno.io/v1/ImageValidatingPolicy \
--api-versions aquasecurity.github.io/v1alpha1/ClusterComplianceReport
For the Compliance App, an OpenID Connect identity provider that the app can reach, directly or through an HTTP proxy, or local accounts held in a Kubernetes Secret.
Installation
Install the three charts in this order: Kyverno, then trivy-operator, then the Convoy. The Convoy refuses to install until the other two are present. The chart repositories and the credentials are described in Access and credentials.
-
Install Kyverno and trivy-operator, as their pages describe: kyverno and trivy-operator. If you will assess only some namespaces, set each engine’s namespaces now, as Keeping the engines inside the scope describes.
-
Create the release’s namespace and its pull secret. The examples use the release name
compliancein the namespacecompliance.1x-ax.jsonis the credential file written as Creating the pull secret describes, which keeps the token out of your shell history; keep it until step 5:kubectl create namespace compliance kubectl --namespace compliance create secret generic 1x-ax \ --type=kubernetes.io/dockerconfigjson \ --from-file=.dockerconfigjson=1x-ax.jsonDo not create the node collector’s namespace (
compliance-node): the chart creates it, and the install stops if it already exists. -
Write a values file with at least the regimes your licence grants:
regimes: - nca-eccThe Parameters list every other setting, such as
scope.namespaces, the schedule, email and object storage. -
Install the chart:
helm install compliance compliance-convoy/compliance \ --namespace compliance --version <chart version> -f <values file> -
Create the pull secret in the node collector’s namespace, which the install has just created:
kubectl --namespace compliance-node create secret generic 1x-ax \ --type=kubernetes.io/dockerconfigjson \ --from-file=.dockerconfigjson=1x-ax.jsonThen delete
1x-ax.json. Until this secret exists, the node collector’s Jobs cannot pull their image, and a run records the node facts as not read. -
Start a first run and read its report, as step 5 of Activating a regime shows.
Regimes
| Regime | Value | Catalogue | Mapping to NIST SP 800-53 |
|---|---|---|---|
| NCA Essential Cybersecurity Controls, ECC-2:2024 | nca-ecc |
176 requirements | Mamluk’s own |
| Qatar Central Bank, Technology Risks (January 2018) | qcb |
516 requirements | Mamluk’s own |
| UAE Information Assurance Standard 2.1 (November 2025) | uae-ia |
134 requirements | The standard’s own, from its Annex D |
| National Information Assurance Standard, version 2.1, issued by the National Cyber Security Agency (May 2023) | qatar-nia |
356 requirements | Mamluk’s own |
| SAMA Cyber Security Framework, version 1.0 (May 2017) | sama-csf |
409 requirements | Mamluk’s own |
| Central Bank of the UAE, Operational Risk Management Regulation | cbuae-orm |
63 requirements | Mamluk’s own |
| Monetary Authority of Singapore, Technology Risk Management: Notices FSM-N05 and FSM-N06 (May 2024) and Guidelines (January 2021) | mas-trm |
259 requirements: 15 from the notices, binding, and 244 from the Guidelines, guidance | Mamluk’s own |
| Central Bank of Iraq, Cyber Resilience Controls, first edition (June 2024), and Banking Reform Standards B2, B3, B7 and B9 (2025) | cbi-crc |
524 requirements: 434 from the Controls, for all banks and payment providers, and 90 from the Standards, for private banks | Mamluk’s own |
Each report states whose mapping of the regime to NIST SP 800-53 it uses. Where the regulator publishes none, the mapping is Mamluk’s own.
MAS Technology Risk Management (mas-trm). One regime holds the two
notices and the Guidelines. Each requirement shows its legal force: a notice’s
paragraph reads “Notice (binding)”, with the notice’s number and a space before
its ID (for example N06 4.6); a Guideline paragraph reads “Guidelines
(guidance)”, with the Guidelines’ own paragraph number. In OSCAL the
same distinction is the applicability property, always for a notice and
risk for a Guideline paragraph. The notices to licensed insurers (FSM-N03,
FSM-N04) and to capital markets intermediaries (FSM-N21, FSM-N22) state the same
requirements; the report uses the bank notices’ numbering for all.
Central Bank of Iraq (cbi-crc). One regime holds the Cyber Resilience
Controls and the technology standards B2 (core banking), B3 (online banking), B7
(infrastructure and data) and B9 (business and operational resilience). Each
requirement shows the text it comes from: “Controls (all banks and payment
providers)” or “Standards (private banks)”; in OSCAL the applicability
property is always for a Controls requirement and risk for a Standards
requirement. The Controls are published in Arabic only, and the Standards in
Arabic with an English reference translation; the summaries in the report are
Mamluk’s own English paraphrase. A Controls ID carries a function prefix (for
example PR-9.1a) and a Standards ID the standard and item (for example
B7 B.6); the prefixes and the letters added to the CBI’s numbers are
Mamluk’s. The data centre and disaster recovery requirements (in Iraq, the
recovery site at least 75 km away) are the bank’s to evidence. Evidence stays on
the cluster’s volume unless delivery by S3 or by email is configured; an
endpoint, mailbox or mail relay outside Iraq places the evidence outside Iraq, so
set s3.endpoint to a store in Iraq.
Regimes and their authorities
Each regime’s report, PDF, email, OSCAL document and the Compliance App carry the
statement below on the regime’s authority, followed by the regime’s notes on its
texts: who owns them, the language they are published in, that the requirement
summaries are Mamluk’s paraphrase, and whose mapping to NIST SP 800-53 the
report uses. In the OSCAL document both are in the metadata remarks, and each is
also a back-matter resource of its own, selected by the property resource
(in the namespace tersane.bahriya.cloud/ns/oscal, which the document writes with the https scheme) with the value statement
or regime-notes. The Compliance App shows the notes on each run’s page.
| Regime | Authority | Statement |
|---|---|---|
nca-ecc |
National Cybersecurity Authority (NCA), Kingdom of Saudi Arabia | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the National Cybersecurity Authority (Saudi Arabia). |
qcb |
Qatar Central Bank (QCB), State of Qatar | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by Qatar Central Bank. |
uae-ia |
UAE Cyber Security Council, United Arab Emirates | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the UAE Cyber Security Council. |
qatar-nia |
National Cyber Security Agency (NCSA), State of Qatar | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the National Cyber Security Agency (Qatar). |
sama-csf |
Saudi Central Bank (SAMA), Kingdom of Saudi Arabia | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Saudi Central Bank. |
cbuae-orm |
Central Bank of the UAE, United Arab Emirates | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Central Bank of the United Arab Emirates. |
mas-trm |
Monetary Authority of Singapore (MAS), Republic of Singapore | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Monetary Authority of Singapore. |
cbi-crc |
Central Bank of Iraq (CBI), Republic of Iraq | Neither Mamluk LLC-FZ nor its products are affiliated with or endorsed by the Central Bank of Iraq. |
Corrections to earlier reports
Multi-factor authentication as the organisation’s evidence (this chart release). Requirements that ask for multi-factor authentication are now the organisation’s to evidence. Earlier, NCA ECC 2-2-3-2, Qatar NIA AM 36, QCB 8.10.5.2 and SAMA CSF 3.3.5.4.e, 3.3.5.4.f.1.a and 3.3.5.4.f.1.b could show Pass, with partial coverage, when sign-in to the cluster was delegated to the organisation’s identity provider, although the Convoy does not test whether that provider requires a second factor. They now show the organisation’s evidence, with the reason. QCB 8.10.2.7 and UAE IA T5.2.3 keep their verdicts on the authentication and access controls the cluster does evidence, and leave the second factor to the organisation, as their scopes state. If an assessor has read an earlier report, its Pass for those six requirements should be read as evidence that sign-in is delegated to the identity provider, and nothing more.
The audit log directory as a check of its own (an earlier chart release). The collector of that release changed what the following show.
- No Pass without the audit log’s evidence. Earlier, the node file permissions check also judged the API server’s audit log directory, and passed on its other files where that directory could not be examined. A requirement resting on it could then show Pass although the audit log was never looked at. The directory is now judged by the audit log permissions check, which is Not evaluated, with the reason, wherever the directory is not examined. See Node facts.
- Requirements that now rest on the audit log’s protection. QCB 8.4.4 (logs protected from change or deletion) and UAE IA T3.2.2 (critical logs protected) gain the check. QCB 8.4.14 and 9.1.1.10, and SAMA CSF 3.3.14.4.h and 3.3.15.4.g, which ask for logs to be protected, were evidenced by the organisation alone; the cluster now evidences the protection of the API server’s audit log, and the organisation the rest, as each scope states. Each can now Fail where the audit log is open to other users, or be Incomplete where it cannot be examined, including on a Bahriya Kubernetes Engine release that does not yet set these modes, where upgrading resolves it.
- The hardening requirements no longer judge the audit log. NCA ECC 1-6-2-2, 1-6-3-5 and 2-3-2, QCB 9.1.1.8, UAE IA T3.1.1 and SAMA CSF 3.3.8.6.a are judged on the Kubernetes configuration, credentials and keys as before, without the audit log directory, so its modes no longer make them fail.
- NCA ECC and CBUAE. Neither names a requirement for the protection of logs in the scope the Convoy assesses, so neither regime judges the audit log’s protection. Their other audit log checks, logging and its retention, are unchanged.
- Runs written before that release read as before.
What each check establishes, and node facts in a scoped run (an earlier chart release). The collector and the Compliance App of that release changed what the following show.
- What each check establishes. Each check in
results.json, the report and the Compliance App’s requirement screen now carries a sentence saying what it establishes, from the regime’s evidence model. Each run also records, asrun.model, the SHA-256 of the model it judged from, so a stored run’s wording and verdicts can be traced to the exact model. No verdict changes. Runs written before this release read as before, without the sentence. - Node facts in a scoped run. In a scoped run that does not list
kube-system, the node checks that need nothing fromkube-systemare now judged, where earlier every node check was Not evaluated. The etcd data directory check is Not evaluated, and each requirement evidenced by the node file permissions check says beside its result that the audit log directory was not examined. A run of the whole cluster, or one that listskube-system, is judged as before. See Scoped assessment.
Scoped assessment, segmentation in meshed namespaces, the audit log directory and the outcome value in OSCAL (an earlier chart release). The collector and the Compliance App of that release changed what the following show.
- Scoped assessment. The Convoy can assess only the namespaces you list. Existing installations assess the whole cluster, as before. See Scoped assessment.
- Organisational evidence in OSCAL. In the OSCAL output, a requirement the
organisation evidences now has the outcome
organisational-evidence, as itsoutcomeproperty and its status reason. Earlier output gavecustomer. Tooling that matches on that value needs the new one.results.jsonkeeps the valuecustomer, so runs written before this release read as before. The emailed summary counts these requirements as “organisational evidence”. - Audit. A policy’s action may be
Audit: the result is recorded, and no warning is returned to whoever applied the object. - Segmentation in meshed namespaces. The namespace NetworkPolicy check no longer passes for a namespace in a service mesh whose traffic can be pooled, where a NetworkPolicy change may not end connections already open. Such a namespace is now Not evaluated, so a requirement that passed through it is now Incomplete: NCA ECC 2-5-2, 2-5-3-1, 2-5-3-2 and 2-5-3-5; QCB 8.1.7, 8.10.1.9, 9.1.1.3, 9.1.1.7, 9.7.1.2 and 9.7.2.1; UAE IA T4.1.1 and T4.1.3; Qatar NIA NS 5, SS 4, SS 21 and SS 23; and SAMA CSF 3.3.7.4.g and 3.3.8.6.e. A cluster with no mesh is judged as before. See NetworkPolicies.
- The audit log directory. The node file permissions check now also judges, on each control-plane node, the API server’s audit log directory and the files in it. A cluster where they are open to other users now Fails that check. See Node facts.
Organisational evidence in the app, and three corrections (an earlier chart release). The collector and the Compliance App of that release changed what the following show.
- Audit-log retention. The audit-log retention check no longer passes when a file-count limit can remove logs sooner than the regime’s period. Such clusters now show Not evaluated, with the central log store as the evidence. The API server deletes a rotated log at its age limit or its file-count limit, whichever comes first, so the age limit alone is no retention period. A cluster whose age limit is below the period still Fails.
- NCA ECC cryptography. NCA ECC 2-8-2, 2-8-3-1 and 2-8-3-3 are now reported as organisational controls, because a platform check cannot yet confirm that only NCS-approved algorithms are negotiated, in transit and at rest. The kubelet cipher check’s fix now names AES-GCM suites only.
- When an assessment ended. Each assessment result in the OSCAL output now ends when the run ended. Earlier output gave the start time as the end.
- Organisational evidence in the app. For each organisational or partly covered requirement, the app keeps comments, attached files, a named owner and a review date, and shows a notice when a review date has passed or a regime’s newest run is overdue. Files are scanned on upload; evidence never changes a verdict; the signed export carries every file. See Organisational evidence in the Compliance App.
- Each regime’s statement. The report, the PDF, the email, the OSCAL output, the app’s screens and its signed export carry the regime’s statement on its authority, shown above.
Proof that logs arrive, and the control plane read directly (an earlier chart release). The collector of that release changed what the following checks showed.
- Log forwarding passes only when the records arrive: Fluent Bit’s own counters for its outputs to a central store show records delivered since the last run, with no error, failed retry or dropped record. Earlier reports passed a configured output whether or not anything reached the store. A delivery that fails, or an output that delivered nothing, now Fails, naming the Fluent Bit Pod; where the counters cannot be read, the check is Not evaluated, and says why. The collector reads the counters directly from each Fluent Bit Pod on its HTTP server’s port, which its NetworkPolicy admits, and is granted no Pod proxy.
- Platform settings on a cluster whose control plane runs as static Pods, as kubeadm and kind run it, and as the Bahriya Kubernetes Engine did before release 2.3: audit logging and its retention, anonymous access, encryption of Secrets at rest and the control-plane TLS settings are read from the API server’s own settings, and judged. A setting that is off is Not provided. Earlier reports left them to the organisation. The other platform settings, and every platform setting on a managed control plane such as EKS, AKS or GKE, remain organisational evidence.
- The Compliance App stops when a database migration meets an object the migrations did not make, as before, and now finds it from the migration’s own failure rather than from the wording of a warning.
A report everyone reads the same way (an earlier chart release). The collector and the Compliance App of that release changed what the following requirements show.
- Organisational evidence. Requirements the organisation evidences
outside the cluster are now called Organisational evidence. Earlier
reports called them Customer evidence; runs store them under the same key,
customer, so earlier runs and OSCAL output read as before (OSCAL output names themorganisational-evidencefrom a later release, above). Organisational requirements show no coverage, which applies only to what the cluster evidences. - Nothing to check. A new outcome, for a cluster that has nothing of the kind a check judges outside the exempt namespaces yet: no application workload, namespace or Ingress, or no backed-up volume. It is neither a pass nor a failure. Earlier reports showed these as Not evaluated.
- Platform requirements on a cluster that declares no platform settings. On a cluster that is not a Bahriya Kubernetes Engine cluster, and on one before release 2.3, the platform requirements are Organisational evidence, technical and outside the cluster: the control-plane settings are evidenced by the cloud provider, or whoever runs the control plane, for example with an attestation or a configuration export. The report gives the reason. Earlier reports showed them as Not evaluated. Where the platform settings could not be read on a BKE cluster, they are still Not evaluated.
- Backups. Velero is judged as Longhorn is: an available backup storage location and an active schedule. Backups pass when either is configured to back up. A cluster with neither Longhorn nor Velero now Fails; earlier reports showed Not evaluated.
- Log forwarding. A cluster with no log shipper now Fails. Vector,
Fluentd, Promtail, Grafana Alloy, Filebeat, the OpenTelemetry Collector, the
Datadog Agent, and Fluent Bit outside
logging, are named, and the check is Recorded: where they send logs is the organisation’s to evidence. - The namespace
defaultis judged by the policies only while it holds a workload, andnetdata, a namespace of the Bahriya Kubernetes Engine, is exempt. - NIST SP 800-53 controls by name. Each control shows its title, and the report lists NIST’s statement of each with a link to NIST. QCB 8.2.2 named IR-10 and UAE IA T7.4.2 named SA-15(9), both of which NIST has withdrawn; they now name IR-4(11) and SA-3(2), which NIST incorporated them into. Both are organisational, so no outcome changes.
Images not scanned (an earlier chart release). The collector of that release changed the vulnerability and operating-system checks, and the requirements that follow them.
- The vulnerability and operating-system checks are now Not evaluated while any workload outside the exempt namespaces has not been scanned by trivy-operator, and they name those workloads. A failure among the images that were scanned still fails. Earlier reports passed them on the workloads that had been scanned, so NCA ECC 2-3-3-3 and QCB 8.3.2 and 9.1.14.7, which name the vulnerability check without the scan freshness check, could show Pass while the scanner had stopped or had not yet reached the cluster’s workloads. A cluster with no workload outside the exempt namespaces passes neither check either.
Three corrections to the collector (an earlier chart release). The outcomes of the following change; every other outcome is as before.
- Log forwarding on a Bahriya Kubernetes Engine cluster. The previous
collector read Fluent Bit’s configuration only from a ConfigMap named
fluent-bit. The Bahriya Kubernetes Engine names itbke-fluent-bit, after its Helm release, so on its clusters the check said Fluent Bit was not installed. The collector now reads the ConfigMap that Fluent Bit’s DaemonSet inloggingmounts, and may read any name influentBit.configMaps(bke-fluent-bitandfluent-bitby default). QCB 8.4.10 and UAE IA T3.2.3 follow the check. - Operating-system support with nothing to judge. Where no image outside the exempt namespaces runs on an operating system trivy recognises, the check passed with nothing judged. It is now Not evaluated, as documented. QCB 8.8.4 follows the check.
- Longhorn or Fluent Bit absent. Kubernetes refuses a read the collector has no rights for before it says whether the object exists, and the chart grants those rights only in namespaces that exist. On a cluster without Longhorn or Fluent Bit, the checks therefore said their configuration “was not read in this run”, as if a read had failed. They now say the tool is not installed, decided from the API’s discovery document and the namespace list. A refusal where the tool is installed is still reported as a fault.
A review of every requirement’s class (an earlier chart release). Each requirement of NCA ECC, QCB and UAE IA was read again against the checks its NIST controls reach. The outcomes of the following change; every other outcome is as before.
- QCB 9.1.13.2 to 9.1.13.5 concern virtual machines, hypervisors and their management. Earlier reports judged them by the cluster’s own hardening, access control, patching and audit log, which say nothing of the virtualisation layer beneath it. They are now Organisational evidence. The cluster’s own results remain under QCB 9.1.1.8, 8.10.1.2, QCB 8.3.2 and QCB 8.4.4.
- QCB 9.1.1.6 (default accounts disabled) now has a verdict: whether anonymous requests are refused and whether Pods receive the default service account’s token. Earlier reports showed it as Organisational evidence.
- QCB 8.10.4.2 now takes its verdict on “logged” from the API server’s audit log. Earlier reports showed it as Recorded.
- UAE IA T3.1.8 (software installed only by authorised administrators) now has a verdict: whether the cluster runs only images from the allowed registries, signed by a configured key. Earlier reports showed it as Organisational evidence.
- NCA ECC 1-9-5 (access rights reviewed when employment ends) now records the cluster’s role bindings for the review. Earlier reports showed it as Organisational evidence.
- Seven requirements on backups and on keeping logs centrally are now judged by the new checks below: NCA ECC 2-9-2 and 2-9-3-1, QCB 8.7.2, QCB 8.7.3 and QCB 8.4.10, and UAE IA T3.1.7 and UAE IA T3.2.3.
If an assessor has read an earlier report, its results for QCB 9.1.13.2 to 9.1.13.5 should be read as evidence about the cluster, not about the virtualisation layer, which remains the bank’s to evidence.
QCB 8.3.6, QCB 8.4.8, QCB 8.8.4, 8.10.1.6, 8.10.4.4 and 9.1.1.5. Reports made before this correction showed QCB 8.4.8, QCB 8.8.4, 8.10.1.6, 8.10.4.4 and 9.1.1.5 as Pass in error, and judged QCB 8.3.6 by whether any vulnerability was open. Each now shows Recorded.
- QCB 8.4.8 asks for a six-monthly review of user access, 8.10.1.6 for a periodic audit of accounts against current roles, and 8.10.4.4 for a current list of privileged users. The report records the cluster’s role bindings for each, but no rule can judge from them whether the review took place. The Pass came from a different check: that people sign in through an identity provider. That is true, but it does not evidence a review.
- QCB 8.3.6 asks for the delay in patching to be measured against the bank’s baseline. The report now records the age of each open vulnerability, and no longer gives a verdict from whether any vulnerability is open.
- QCB 8.8.4 asks for end-of-life and end-of-support dates to be tracked. The Pass came from the scans of the workloads’ images, which judge whether critical or high vulnerabilities are open, not whether an image’s operating system is still supported. The report now records the versions of the Bahriya Kubernetes Engine and Kubernetes that the cluster runs, and no check judges support. Two checks now judge it; see End-of-support checks.
- 9.1.1.5 asks for infrastructure and network devices to be tested for vulnerabilities. The Pass came from the scans of the workloads’ images, which do not reach the control plane, the platform or network devices. The report now records the versions of the Bahriya Kubernetes Engine and Kubernetes that the cluster runs, and its inventory.
If an assessor has read an earlier report, its Pass for QCB 8.4.8, 8.10.1.6 and 8.10.4.4 should be read as evidence that people sign in through the organisation’s identity provider, and its Pass for QCB 8.8.4 and 9.1.1.5 as evidence that the workloads’ images carry no open critical or high vulnerability, and nothing more. The review, the testing and the tracking of support dates remain the bank’s to evidence.
At the same time, three requirements on multi-factor authentication (NCA 2-2-3-2, QCB 8.10.2.7 and 8.10.5.2) gained a scope. It states that the evidence is the delegation of sign-in to the identity provider, and that whether that provider requires a second factor is the organisation’s to evidence. Eight requirements whose verdict comes from the workloads but which name systems, devices, databases or assets beyond them (NCA 2-1-5 and 2-3-3-3; QCB 5.1.4, QCB 8.3.2, QCB 8.8.4, 9.1.13.4, 9.1.14.7 and 9.4.2.3) gained a scope saying what the verdict covers. Four requirements on identification and remote access (QCB 8.10.2.1, 8.10.2.2, 8.10.5.3 and 9.10.1.9) gained a scope saying that the verdict concerns sign-in through the identity provider, and that shared credentials such as kubeadm’s admin.conf, and SSH access to the nodes, are the bank’s to evidence. Their outcomes are unchanged.
End-of-support checks
QCB 8.8.4 carries a verdict again, from two checks:
- Platform releases. The Kubernetes release and the Bahriya Kubernetes Engine release that the cluster runs are compared with their published support periods. A release out of support fails.
- Image operating systems. Each workload image’s operating system, as trivy-operator’s vulnerability scans record it, fails if it is past its end of service life. Images with no operating system that trivy recognises (scratch and static images, and operating systems trivy does not know) do not block the check. The report lists each of them by name, so that they can be reviewed by hand. Their packages remain judged by the vulnerability check.
QCB 8.8.4 shows Incomplete only where trivy-operator has reported no vulnerability scan. No new scan is needed after the upgrade, because the existing scans already record each image’s operating system. NCA ECC results are unchanged.
Backup and log forwarding checks
Three checks read the cluster’s backups and its logging:
- Backups. Longhorn has a backup target that it reports available, and a
recurring backup job, such as the
bke-daily-backupjob that the Bahriya Kubernetes Engine creates; or Velero has a backup storage location that it reportsAvailable, and a schedule that is enabled and not paused. Either is enough. The last backup of each volume and each schedule, and its age, are recorded beside the verdict. With neither tool installed, the check Fails. - Backup encryption. Every backed-up Longhorn volume outside the exempt namespaces is encrypted, so that its backups leave the site encrypted. The encrypted StorageClasses are listed beside the verdict. Velero’s backups are encrypted, or not, by the object store it writes to, which the cluster does not show: with Velero alone the check is Not evaluated, and records which server-side encryption settings each storage location names. With no backup tool, nothing leaves the site, and there is Nothing to check.
- Log forwarding. An output of Fluent Bit’s configuration sends to a host,
or an
s3output to a bucket, and the records arrive: Fluent Bit’s own counters for those outputs, read on each of its Pods, show records delivered since the last run, or since the Pod last started, with no error, failed retry or dropped record. Outputs tostdout,null,counteror afileon the node do not count. A failed delivery, or an output that delivered nothing, Fails, naming the Pod and the output; where the counters cannot be read, the check is Not evaluated. Another log shipper (Vector, Fluentd, Promtail, Grafana Alloy, Filebeat, the OpenTelemetry Collector, the Datadog Agent, or Fluent Bit outsidelogging), recognised by its image, is named and Recorded: its configuration is not read, so where it sends logs is the organisation’s to evidence. With no log shipper, the check Fails.
Where Longhorn or Fluent Bit is not installed, the report says so. Fluent Bit’s configuration is the ConfigMap its
DaemonSet in logging mounts, bke-fluent-bit on a Bahriya Kubernetes Engine
cluster; a release of another name is added to fluentBit.configMaps. From
Fluent Bit’s configuration the collector keeps only each output’s plugin, host
and bucket; its credentials are read with it, dropped at once, and written
nowhere. Fluent Bit’s counters are read directly from each Fluent Bit Pod’s
HTTP server, which its chart serves on port 2020 by default
(fluentBit.metricsPort). While the collector delivers by email or to S3 and
its NetworkPolicy applies, that policy admits this one port, on Pods labelled
app.kubernetes.io/name: fluent-bit in logging, and nothing else in that
namespace. A read that is refused or times out leaves log forwarding not
evaluated, with the reason; it is never passed. The collector keeps the
counters and nothing else. From a backup target it keeps only the scheme of its address; from a
Velero storage location, its provider and the names of its encryption
settings, not its bucket or their values. It reads no Secret.
Activating a regime
Each regime is licensed on its own, and the Convoy is licensed with the first regime. There is no licence for every regime: a licence grants the regimes it names, and no others. To add a regime to a running release:
-
Have the regime added to the licence. Ask Mamluk to add the regime to your organisation’s Bahriya Tersane licence: in the Mamluk console, open Support, choose Raise a ticket, choose “Licences and clusters” under “What is this about” and “Bahriya Tersane and its Convoys” under “Which product”, and name the regime; or ask your organisation’s contact at Mamluk. Any member of the organisation may raise a ticket. Mamluk changes the licence; the console shows it, but cannot change it.
-
Confirm the grant in the console. On the organisation’s Tersane page in the Mamluk console, the licence names the Convoy with each regime it grants, such as “Compliance and Assurance Convoy (NCA ECC, QCB)”, and the images the licence may pull include the regime’s evidence model:
docker pull 1x.ax/tersane/convoy/compliance/regimes/qcb/catalogue:<tag>The chart repository follows a change to the licence on its next request. The registry follows it within 15 minutes.
-
Check the credentials. No new credential is needed: the organisation’s chart credential and image tokens serve every regime the licence grants. The pull secret named in
collector.imagePullSecrets(1x-axby default) must exist in the release’s namespace and in the node collector’s namespace (nodeCollector.namespace, by default the release’s namespace with-nodeadded). The chart creates the node collector’s namespace, so its pull secret is created after the first install, as step 5 of Installation shows; never create that namespace yourself. Access and credentials describes the credentials and how to create the pull secret. -
Add the regime to the values and upgrade. The chart version must hold the regime: it is listed in the table above in that version’s README, and the chart refuses a regime it does not hold. In the values file you keep:
regimes: - nca-ecc - qcbthen:
helm repo update helm upgrade <release> compliance-convoy/compliance \ --namespace <namespace> -f <values file>The upgrade adds the regime’s CronJob,
<release>-collect-qcb, and its evidence volume,<release>-evidence-qcb, and restarts the web view so that it serves the new volume. -
Start a first run and read its report. Rather than wait for the schedule:
kubectl -n <namespace> create job collect-qcb-now --from=cronjob/<release>-collect-qcb kubectl -n <namespace> wait --for=condition=complete job/collect-qcb-now --timeout=30mThe run’s report is then in the web view, under the regime, and on its volume, and it is sent by email and to object storage where those are configured. A run whose Pod does not start with an image pull error is a regime the licence does not yet grant to the registry, or a missing pull secret: check steps 2 and 3.
Removing a regime. Remove it from regimes and upgrade. The regime’s
CronJob is removed and the web view no longer serves it. Its evidence volume
is kept, with every run on it, so that the evidence remains available; delete
it with kubectl -n <namespace> delete pvc <release>-evidence-<regime> when
it is no longer needed.
When a regime is no longer licensed. Each run pulls the regime’s image, so the registry is asked on every run. A regime that the licence no longer grants stops at the pull: the Pod does not start, the collector does not run, and nothing is written. When one regime is removed from the licence, the others carry on; when the licence is suspended, revoked or expires, every regime stops. The runs already on the volumes are not touched. When the regime or the licence is restored, the next scheduled run proceeds.
Scoped assessment
By default the Convoy assesses every namespace of the cluster. An audit scope is often narrower. List the namespaces in scope, and the Convoy assesses only those:
scope:
namespaces:
- payments
- payments-batch
- kube-system
With a list:
- the collector reads workloads, reports, role bindings, Ingresses and Velero’s objects only in the listed namespaces. Its permission to read them is a read-only Role in each listed namespace; its ClusterRole keeps only the cluster-scoped reads (nodes, namespaces, ClusterRoles and their bindings, the admission policies, cluster-scoped reports and StorageClasses). It cannot read a namespace that is not listed;
- the admission policies and Kyverno policies apply only in the listed namespaces, and still not in an exempt one;
- each listed namespace must exist when the chart is installed.
What the report states. The report’s first page, the PDF, the email, the OSCAL output and the Compliance App’s run screen state the scope in one sentence, for example:
Assessed: the 18 listed namespaces. Not assessed: 4 other namespaces of this cluster. This report is not evidence for them.
The namespaces that were not assessed are counted, never named. Each requirement the cluster evidences also says where its evidence comes from: the listed namespaces only, or the whole cluster, for the checks of the nodes, the platform, backups and log forwarding. A listed namespace that the cluster does not have is named.
List the whole audit scope. A namespace in your auditor’s scope that is left off the list leaves every requirement it bears on unproven. Agree the list with your auditor.
Platform evidence. Some evidence sits in a fixed namespace: the control
plane’s settings in kube-system, the platform’s settings in
platform.namespace (bahriya-system by default), Fluent Bit’s configuration
in logging and Longhorn’s backups in longhorn-system. When that namespace
is not listed, it is not read, and the checks that need it are Not
evaluated. List it to have them judged. The node collector finds the etcd
data directory and the API server’s audit log directory from kube-system.
When kube-system is not listed, it is not read: every other node fact is
gathered and judged, the etcd data directory and audit log permissions
checks are Not evaluated, and each requirement evidenced by the audit
log permissions check says beside its result that the audit log directory
was not examined. List kube-system to have both judged.
Assessing the platform’s own namespaces. By default the admission and
Kyverno policies do not apply in policies.exemptNamespaces, which holds the
namespaces of a Bahriya Kubernetes Engine cluster’s own components. A scope
that lists those namespaces, as an assessment of the platform itself does,
therefore leaves the policies applying nowhere in it. To judge them, remove
the listed namespaces from policies.exemptNamespaces (the list replaces the
default as a whole) and set policies.defaultAction to Audit: Audit
records each result in the policy reports and never warns or refuses a
request. The platform’s components need host access and privileges by
design, so expect many recorded failures there.
Scoping changes nothing commercially: the licence is the same, per regime and per organisation; scope is an assessment setting, not a tier.
Keeping the engines inside the scope
The claim that an unlisted namespace is never read is made for the Convoy’s collector. Kyverno and trivy-operator are separate charts, and by default they work in every namespace. To keep them to the same namespaces, set them when you install them:
-
trivy-operator scans and writes its reports only in the namespaces named in
engine.targetNamespaces, a comma-separated list. Give it the same list asscope.namespaces:engine: targetNamespaces: "payments,payments-batch,kube-system" -
Kyverno applies the Convoy’s policies only in the listed namespaces, because the Convoy’s policies select them, and writes their results only there. Its webhooks can also be limited to chosen namespaces, so that no admission request from any other namespace reaches it, and made to fail open. The kyverno page gives the values.
Both engines keep the cluster-wide read permission their charts give them.
Which reports the collector reads. The Convoy reads its policy results
from Kyverno’s policy reports, PolicyReport and ClusterPolicyReport in
wgpolicyk8s.io/v1alpha2, which Tersane’s kyverno chart writes, and from the
openreports.io/v1alpha1 reports on a cluster that serves that API. A report
type the cluster does not serve is not read. It reads trivy-operator’s
reports in aquasecurity.github.io/v1alpha1. Namespaced reports are read
only in the listed namespaces.
Evidence
One CronJob per regime, <release>-collect-<regime>, runs the evidence
collector (daily at 02:17 by default, collector.schedule). It mounts the
regime’s image as a read-only image volume, pulled on every run, so the
licence is asked each time: a regime that cannot be pulled stops at the pull,
the collector never starts, and the other regimes’ runs carry on. It reads
the cluster with a ClusterRole that may only get and list, Velero’s backup
storage locations and schedules and the cluster’s Ingresses among what it
reads. Where the namespaces exist, it may also read Longhorn’s backup targets, recurring jobs
and volumes in longhorn-system, and the ConfigMaps named in
fluentBit.configMaps in logging, by name (longhorn.createBinding and fluentBit.createBinding
create these where the chart is rendered with helm template). It is granted
no Pod proxy, exec, attach or port-forward. The only
thing it writes through the API is the node collector’s Jobs, in a namespace
of their own (below).
Each run is a directory on the regime’s volume, <release>-evidence-<regime>,
named by its UTC time: the report in HTML and PDF, OSCAL assessment results,
the full results, the snapshot of what was read, and a SHA-256 manifest,
signed when evidence.signingKey.existingSecret names a Secret holding a PEM
private key. Runs older than evidence.retentionDays (400) are removed. The
volume is kept when the regime or the chart is removed.
The report’s “Fix first” section, after the summary, lists every object that
fails a check, grouped by namespace, the most severe first. For each it gives
the requirements it fails, what is wrong, how to fix it, and the kubectl
command that shows the full result in trivy-operator’s or Kyverno’s report.
Fix text for trivy-operator’s checks is trivy-operator’s own, labelled so,
except where the Convoy replaces it; the rest is the Convoy’s. The same
severity, fix text and command are in results.json and in the OSCAL
assessment results.
To start a run now:
kubectl -n <namespace> create job collect-now --from=cronjob/<release>-collect-nca-ecc
What each outcome means
Each requirement has one outcome, and the report, the App and the OSCAL output use the same words:
| Outcome | Meaning |
|---|---|
| Fail | The cluster was checked and something does not meet the requirement. The findings say what and where. |
| Not provided | A control the requirement needs is switched off on this cluster. |
| Incomplete | Everything that could be checked passed, but part of the requirement could not be checked in this run. The reason is given. |
| Not evaluated | Nothing behind this requirement could be checked in this run. The reason is given. It is not a pass. |
| Pass | Every check behind the requirement passed on this run’s date. Where coverage is partial, the pass covers the cluster’s part only. |
| Recorded | Facts collected for the organisation’s reviewer, such as an inventory or a list of access; no verdict. |
| Nothing to check | The cluster has nothing of this kind yet, for example no Ingress to check for TLS. Neither a pass nor a failure. |
| Organisational evidence | The organisation evidences this requirement outside the cluster; the Convoy checks nothing here. |
Each organisational requirement also says what kind of evidence it asks for: technical, outside the cluster (held by IT or security engineering), non-technical (policy, people, premises, law or contracts), or both. The App filters by it: in the cluster, technical, outside the cluster and non-technical. Each NIST SP 800-53 control is named by its title, and the report lists NIST’s statement of it with a link to NIST.
On a cluster that is not a Bahriya Kubernetes Engine cluster, or one before release 2.3, the cluster declares no control-plane settings to the collector. Where its control plane runs as static Pods, as kubeadm and kind run it, the collector reads audit logging and its retention, anonymous access, encryption of Secrets at rest and the control-plane TLS settings from the API server’s own settings, and judges them. The other platform requirements, and all of them on a managed control plane such as EKS, AKS or GKE, are organisational evidence, shown under technical, outside the cluster, with the reason.
Pass, with exception
The Convoy accepts one finding that the engines report on a template which the
running containers do not share. Envoy Gateway writes each Envoy proxy
Deployment with a container named shutdown-manager that has requests and no
limits, and gives no setting for its limits. The resource-limits policy and
trivy-operator’s checks AVD-KSV-0011 and AVD-KSV-0018 judge the Deployment’s
template, so they fail on it, although a LimitRange in the namespace gives the
container its limits when each Pod is admitted. The sipar-gateway chart
installs that LimitRange.
The finding is accepted in a run only when every one of these holds:
- the finding names the container
shutdown-managerand no other container; - its workload owns at least one running Pod that carries Envoy Gateway’s proxy
labels (
app.kubernetes.io/name=envoy,app.kubernetes.io/component=proxy,app.kubernetes.io/managed-by=envoy-gateway, and either bothgateway.envoyproxy.io/owning-gateway-nameandgateway.envoyproxy.io/owning-gateway-namespace, orgateway.envoyproxy.io/owning-gatewayclass); - every such running Pod’s
shutdown-managerhas a CPU and a memory limit; - the collector read the namespace’s LimitRanges, and one sets a
Containerdefault for both CPU and memory.
Anything missing leaves the finding a failure, with the reason beside it. A
check whose only failures are accepted findings shows Pass, with exception
in the report, the email, the web view and the Compliance App, never a plain
Pass; the accepted finding is listed under the check with what was excepted,
why, and the live limits and LimitRange read in the run. Any other failure
keeps the check failed, and the accepted finding is still listed. The
requirement counts as a pass, and the summary shows how many passes have such
a check behind them (“of which with exception”). In the OSCAL output the
observation is satisfied, carries the property exception with the value
shutdown-manager-limitrange, and names the Pods, their limits and the
LimitRange as its evidence.
The collector reads LimitRanges from the chart release that introduced the exception on. A cluster on an earlier chart cannot read them, so the exception is never applied there. The exception is the only one; it is part of the collector and cannot be extended through values. When Envoy Gateway gives a setting for the container’s limits and the template carries them, the checks pass without it.
The web view
The web view serves every regime’s runs from their volumes, read-only, the newest first, with each run’s report, OSCAL assessment results, results, manifest and signature:
kubectl -n <namespace> port-forward service/<release>-web-view 8080:80
then open http://localhost:8080/. It is static files served by nginx: it
writes nothing, holds no credential and no service account token, and has no
authentication of its own.
Who can read it. By default nothing reaches it over the network, so it
is read only through kubectl port-forward, by those whose Kubernetes role
allows pods/portforward in the release’s namespace. To publish it, add an
Ingress of your own and admit your ingress controller in webView.allowFrom;
the Ingress, and the authentication in front of it, are then yours.
A run waits for the web view’s node. Each evidence volume is
ReadWriteOnce, and the web view holds them mounted, so each collector run is
scheduled to the web view’s node and waits while that node is unavailable.
webView.enabled=false removes the web view and the constraint.
The Compliance App
complianceApp.enabled=true serves the reports, the requirements and what to
fix first behind your identity provider or local accounts, with an access log, on the same
evidence volumes. It takes the web view’s place. It reads the evidence
read-only, holds no service account token, and needs nothing from the Kubernetes
API. Install the chart once on a cluster: its policies have fixed names, and
Helm refuses a second installation.
complianceApp:
enabled: true
oidc:
issuer: https://idp.example.com/realms/example
clientId: compliance
audience: compliance
jwksUri: https://idp.example.com/realms/example/protocol/openid-connect/certs
egressCidrs: ["192.0.2.10/32"]
roles:
- {group: platform-admins, role: administrator}
- {group: auditors, role: auditor}
- {group: team-a, role: namespace-owner, namespaces: [team-a]}
ingressFrom: [ingress-nginx]
With the identity provider, the chart refuses to render without the issuer, the
client, the audience, the address of its keys, its network addresses and an
administrator mapping. The address of its keys, oidc.jwksUri, is the jwks_uri
in its discovery document, at https://<issuer>/.well-known/openid-configuration.
Publish the Service compliance-app, port 80, with an Ingress of your own and
your certificate.
Signing in. complianceApp.auth.mode is oidc (the default), through your
identity provider; local, with local accounts; or both, each person through
one of the two. See Local accounts.
What the app may reach. With the identity provider, a NetworkPolicy limits it
to DNS and to oidc.egressCidrs on port 443. The value is required and has no default.
["0.0.0.0/0"], for a provider whose addresses change, means the app may open
connections to any address on that port: that nothing leaves the cluster then
rests on the app’s code and not on the network.
With local accounts alone, the NetworkPolicy allows no egress at all, not even DNS.
Through a proxy. Where a proxy is the only way out of the network, set
oidc.httpsProxy to its address, http://host:port. The app’s one outbound
request, for the provider’s keys, then goes through the proxy, which tunnels it
to the provider with CONNECT: TLS runs from the app to the provider, and the
provider’s certificate is checked as before. oidc.egressCidrs and
oidc.egressPort then name the proxy’s addresses and port, so the app reaches
nothing but the proxy, and the proxy decides where traffic goes. People’s
browsers still reach the provider for sign-in over their own network.
complianceApp:
oidc:
httpsProxy: http://proxy.example.com:3128
egressCidrs: ["192.0.2.20/32"]
egressPort: 3128
proxyCredentialsSecret: compliance-app-proxy
If the proxy needs a username and a password, create a Secret with them under
the keys username and password, and name it in
oidc.proxyCredentialsSecret. They are sent to the proxy alone and never logged.
oidc.noProxy lists hosts fetched directly despite the proxy, separated by
commas. The app reads no other proxy setting, so a proxy configured elsewhere in
the cluster’s environment is not used. The chart refuses an https:// or
socks5:// proxy, an address that holds credentials, and a credentials Secret
without a proxy.
Local accounts
Local accounts sign people in with a username and a password, checked against hashes held in a Kubernetes Secret. They are for evaluation, and for use before your identity provider is connected. What they lack:
- no multi-factor authentication: a password alone signs a person in;
- no central offboarding: a person who leaves keeps their account until it is removed from the Secret; disabling them in your directory does nothing here;
- no password reset, expiry or self-service: the administrator changes the Secret.
The app’s settings and role-mapping screens state the same. With both, local
accounts sit beside your identity provider: they bypass it, its multi-factor
authentication and its offboarding, and they still sign in when it cannot be
reached. Each local account appears with the prefix local: in the access log
and on every screen, and its role comes from the Secret alone, never combined
with a group mapping.
The Secret has one key, users.yaml. Each account has a hash, one role, for
namespace-owner its namespaces, and an optional expiresAt (a date is the last
day it signs in). It must hold at least one administrator.
users:
a.rahman:
hash: "$2y$12$..."
role: administrator
payments-team:
hash: "$2y$12$..."
role: namespace-owner
namespaces: [payments]
Each hash is bcrypt of cost 12 or more, or argon2id. A plaintext password, or any record the app cannot accept, stops the app at start with a message naming the account and the reason, never the value. Make a hash with the app’s own command, which asks for the password twice without showing it and prints the hash alone:
kubectl run hash-password --rm -it --restart=Never \
--image=<the app's image, as the chart pins it> \
--command -- /var/www/bin/hash-password
or with htpasswd -nbBC 12 <username> <password>, taking the part after the
colon (that puts the password on a command line). Write the file, then create
the Secret from it, so that no hash is typed on a command line:
kubectl -n compliance create secret generic compliance-local-accounts --from-file=users.yaml
complianceApp:
enabled: true
auth:
mode: local
local: {existingSecret: compliance-local-accounts}
ingressFrom: [ingress-nginx]
A change to the Secret needs no restart. Kubernetes updates the mounted file
within its sync period, and the app takes it up at the next request after that:
a removed account is refused, and its sessions end; a changed role is in force.
On a kind test cluster a removed account was refused 68 to 85 seconds after the
Secret was changed, in three runs. kubectl rollout restart
deployment/compliance-app makes it sooner. A change the app cannot accept is
refused, the accounts already in force stay, and the refusal is in the Pod’s log.
Sessions. A correct password gives a 15-minute access token, signed by the
app, which the browser renews while the session is active and holds in memory
only; the renewal uses a cookie the page cannot read. The session ends after the
settings screen’s limit without activity, at sign-out, and when the account
leaves the Secret. The key that signs these tokens is generated on the app’s
volume at first start, or is the key signing-key.pem in the Secret
complianceApp.auth.signingKeySecret names. Replacing it ends every local
session. The issuer of these tokens, complianceApp.auth.local.issuer, must
differ from your identity provider’s.
Failed sign-ins. Every attempt is in the access log, with the username tried, the address and the reason; the password never is. Any failure gets the same answer. After 5 failures within 15 minutes, a username, and an address, must wait 30 seconds, doubling with each further failure to at most 15 minutes.
Memory. A report too large for phpMemoryLimit is listed as unreadable, with
its files still available. To read it, raise phpMemoryLimit and
resources.limits.memory together and restart. The chart refuses a container
limit below (workers + 1) x phpMemoryLimit + 256Mi, under which a large report
would have the container killed and would not fail with an error.
The access log’s key. Each entry is signed with a key held in a Secret. Helm
generates it at installation and keeps it, across upgrades and after the chart is
removed. Where the chart is rendered with helm template, as GitOps tools do,
you must set complianceApp.accessLogKeySecret. The chart cannot read the
cluster there, so without it each render generates a new key and the tool applies
it. The chart cannot tell such a render from a first installation, and does not
refuse it. Create the Secret first, and keep it outside the rendered manifests:
kubectl -n <namespace> create secret generic compliance-access-log-key \
--from-literal=access-log-key="$(openssl rand -hex 32)"
To keep a key Helm has generated, name its Secret, <release>-app-signing.
If the key changes, the entries already written are not lost, and they are not passed as good: the app has no other copy of the old key, so each is marked “Does not match its signed record”, with a notice above the access log that names both causes, an edit or a changed key. The mark itself does not tell them apart. Restoring the old key and restarting the app clears the mark on those entries, and it then falls on what the other key signed. No alarm is raised. After changing the Secret’s contents, restart the app, which reads the key when it starts.
What the signature shows, and what it does not. Each entry is signed on its own. An entry edited in the database shows; an entry deleted from it does not, and leaves no trace in the others. The Pod’s log holds the same records, so collecting it in your own log system gives you a copy to compare. The app’s maintenance checks the newest 200 entries every 5 minutes, and a mismatch appears on the administration pages and as an error line in the Pod’s log.
A run waits for the app’s node, as it does for the web view’s.
Monitoring. Port 9100 serves /metrics, with no token. It is not on the
Service, and the NetworkPolicy admits only the peers in
complianceApp.metrics.allowFrom, none by default. A Netdata collector on the
same node reaches it as the node, which a NetworkPolicy does not stop. The Pod
carries the prometheus.io scrape annotations. The name of the Deployment,
compliance-app, and of its container, web, are fixed because Netdata’s alarms
name them.
Alarms on a Bahriya Kubernetes Engine cluster. Current BKE releases give
Netdata five alarms on the app; BKE’s release notes name the release that
added them, and earlier releases have none of them. The
alarms are evaluated on the cluster’s Netdata parent. compliance_app_metrics_writes raises when
the app stops writing its metrics, and compliance_app_metrics_scrape when
Netdata, while it is itself running and connected, cannot read them.
compliance_app_run_stale raises when a regime has no run as
recent as collector.schedule and collector.activeDeadlineSeconds allow: 24.5
hours on the default daily schedule, or complianceApp.runStaleAfterHours. A
suspended CronJob therefore raises it. compliance_app_storage_level warns as a
volume fills and is critical near full. compliance_app_not_ready raises when the
Deployment has had no ready replica for three minutes.
- Delivery is off until you turn it on. Netdata shows the alarms, but they reach a person only once a notification channel is turned on in BKE’s Netdata health settings, where BKE ships every channel off, or the cluster’s Netdata is connected to Netdata Cloud with notifications set there. Once a channel is on, Netdata’s own stock alarms are sent on it as well.
- A run that fails before it can record why appears only through
compliance_app_run_stale, so up to 24.5 hours later on the daily schedule. This happens, for example, when the collector cannot start or is killed for memory. A run that fails after it starts records why, and that appears on the Settings screen when the run ends. - What raises no alarm: the Deployment deleted; Netdata’s
k8s-statecollector or its parent down; and a Ready Pod without the scrape annotations. Each of these is silent, so check them when you change the cluster or Netdata. - While Netdata’s agent on a node is disconnected from its parent, the parent shows none of that node’s alarms, these included, and in testing they had not reported again several minutes after the agent reconnected. Do not rely on these alarms alone to tell you that the app’s metrics cannot be collected.
- The alarms rely on Netdata’s collector on the same node reaching the Pod as that node, as described under Monitoring above.
When the evidence is overdue. The app and the alarm compliance_app_run_stale
use one rule. A regime’s newest run is overdue once more time has passed since it
started than collector.schedule and collector.activeDeadlineSeconds allow,
24.5 hours on the default daily schedule, or complianceApp.runStaleAfterHours.
The app then shows the auditor and the lead a notice against that regime on the
Regimes list and on its newest run: no run has completed since that run, by when
one was expected, and, where the last attempt failed, when it failed. The reason
for a failure is shown only to administrators, on the Settings screen. Times are
shown in UTC. A schedule that cannot be read is judged on a daily gap, and the
Settings screen says that it cannot be read.
With no run yet, the time runs from the app’s first start, which the app records
once on its own volume, <release>-app-data. If that record cannot be written or
read, a regime with no run is shown as overdue, and the reason is in the Pod’s log.
The record is published with a hard link, so that two processes starting together
record one time: the volume’s filesystem must support hard links, as Longhorn’s
volumes and local volumes do.
When a process stops. Beside the web server, the container runs a process
that writes the metrics and one that maintains the database. Each is started again
if it stops. If one hangs, or cannot be started again, the alarm
compliance_app_metrics_writes raises, and the container is restarted some
minutes later: for the metrics, the alarm after about 2 minutes and the restart
after about 6; for the maintenance, the alarm about an hour after its last pass
and the restart at about 66 minutes. You see the alarm raised and then cleared, an
event Liveness probe failed on the Pod that names the process, and a higher
restart count. The access log and the run index are on the app’s volume, and the
evidence is not changed. Where the alarm returns after each restart, read the Pod’s
log and the Settings screen.
Organisational evidence in the Compliance App
The cluster cannot evidence a requirement about a policy, a committee or a
contract. For each organisational or partly covered requirement, the role
evidence-owner adds the organisation’s own evidence in the app: comments,
attached files, a named owner and a review date. auditor and lead read it;
namespace-owner and administrator do not. It is shown as provided by the
organisation, apart from the outcome, and never changes a verdict, a count or a
file of the collector’s. It carries from run to run.
complianceApp:
roles:
- group: compliance-evidence-owners
role: evidence-owner
evidence:
clamav:
host: clamav.clamav.svc.cluster.local
namespace: clamav
Files are scanned before they are kept. The app needs a clamd it can reach
over clamd’s INSTREAM command. The supported installation is Tersane’s ClamAV
chart at its current version, beside the Convoy: name its Service in
complianceApp.evidence.clamav (host, port and namespace). That chart
runs ClamAV as a non-root user with CPU and memory limits, so it runs in a
namespace that enforces the restricted Pod Security Standard and in one whose
quota requires limits. Each file is scanned by clamd before it is accepted; an
infected file, or a scan that cannot be made, is refused, nothing is stored, and
the access log records the reason. Each accepted file records the version and
date of the signatures it was scanned with, on the item, in the access log and
in the export. While those signatures are older than
complianceApp.evidence.clamav.maxSignatureAgeDays (7 days by default), files are
refused, and the refusal gives their date: signatures that do not update usually
mean freshclam cannot reach its updates, so check the ClamAV chart’s proxy and
private mirror settings. complianceApp.evidence.maxFileMiB must not exceed the largest
stream the scanner accepts (clamd’s StreamMaxLength); a larger file is refused
by the scanner. Without a scanner, or where it cannot be
reached, the app says so on the evidence screen and in each refusal, names the
chart and the values to set, and comments, the owner and the review date can
still be recorded. With one, the app’s NetworkPolicy admits DNS and
the ClamAV Pods on clamd’s port, in that namespace, and nothing else of the
cluster, with local accounts as with an identity provider.
What can be attached. PDF, PNG, JPEG, plain text, CSV, and Word, Excel and
PowerPoint documents, judged by their content as well as their name, up to
complianceApp.evidence.maxFileMiB (25 MiB) each. No archives, and no document
with macros. A file is always downloaded as an attachment, never shown in the
browser. Files are kept on their own volume, <release>-app-files
(complianceApp.evidence.storage, 5 GiB), which stays when the chart is removed.
Removal and reviews. A removed item stays listed, marked removed, with who removed it and when; its file is deleted once it has been removed for longer than the access log’s retention (400 days by default). A review date that has passed is flagged on the requirement and counted on the regime. The app sends no email.
The export. An auditor or a lead downloads a run with the organisation’s
evidence as one ZIP file: the run’s files unchanged, with the collector’s manifest
and signature; the organisation’s items, owners and files, with an OSCAL
assessment-results file of their own whose observations are marked as provided by
the organisation; and export-manifest.sha256, the digest of every file, signed
with the app’s own ECDSA P-256 key. The settings screen shows the public key.
Check a package with:
cosign verify-blob --key export.pub --signature export-manifest.sha256.sig \
--insecure-ignore-tlog export-manifest.sha256
sha256sum -c export-manifest.sha256
Helm generates the key at installation and keeps it. Where the chart is rendered
with helm template, as GitOps tools do, set
complianceApp.evidence.exportKeySecret to a Secret of yours whose key
cosign.key is an unencrypted ECDSA P-256 private key in PEM; without it, each
render would generate a new key, and exports already given to an assessor would no
longer verify against the key the app shows.
With email.mode: smtp, each run is sent through your own SMTP server: the
summary and “Fix first” as the message, with the PDF report attached, and the
HTML report too if email.attach names it, up to email.maxAttachmentMB.
email:
mode: smtp
smtp:
host: smtp.example.com
port: 587
tls: starttls # or implicit
existingSecret: smtp # keys username and password
egressCIDRs: [192.0.2.25/32]
from: evidence@example.com
to: [security@example.com]
TLS is required: with starttls a server that does not offer STARTTLS is
refused, and the server’s certificate is always verified, against the
system’s CAs or the CA in email.smtp.caSecret. Nothing is sent while
email.mode is none, the default.
Object storage
With s3.enabled, each run’s files are uploaded to your bucket as
<prefix><regime>/<run>/<file>, over HTTPS with the store’s certificate
verified. s3.endpoint is required while S3 delivery is on, as
https://<host>[:<port>]: an S3-compatible store, such as MinIO or Ceph, or,
for Amazon S3, https://s3.<region>.amazonaws.com. The chart refuses to install
without it and does not choose a provider for you. s3.pathStyle puts the
bucket in the path, as many S3-compatible stores expect. The keys are in the
Secret s3.existingSecret (accessKeyId, secretAccessKey).
Evidence stays on the cluster’s volume unless you configure delivery by S3 or by email. An endpoint, mailbox or mail relay outside the country whose rules apply places the evidence outside it.
What delivery records, and the collector’s traffic
Each delivery is recorded in the run as delivery.json. It is written after
the run’s manifest is signed, so it is not covered by the manifest. A delivery
that fails is recorded and logged, and the run still completes: the evidence
is on the volume whether or not it was sent.
While email or object storage is on, a NetworkPolicy limits the collector’s
traffic to DNS, the Kubernetes API server, Fluent Bit’s HTTP server on
fluentBit.metricsPort (2020) on Pods labelled
app.kubernetes.io/name: fluent-bit in logging, and the addresses you give
in email.smtp.egressCIDRs and s3.egressCIDRs; the chart refuses to install
without them. If your Fluent Bit serves its HTTP server on another port, set
fluentBit.metricsPort to it, or log forwarding is not evaluated. To allow any address, write 0.0.0.0/0 and ::/0.
After adding a control-plane node, upgrade the chart. The API server’s addresses are read from the
kubernetesEndpointSlices indefaultwhen the chart is installed or upgraded. Until the next upgrade, the collector cannot reach the new node’s API server. Where the chart cannot read them, for example when a GitOps tool renders it, give them inapiServer.cidrs.
Storage that binds on first use. Where the storage class binds a volume
only when a Pod first uses it (volumeBindingMode: WaitForFirstConsumer), the
evidence volumes stay Pending until each regime’s first run, and
helm install --wait waits for them until it times out. Install without
--wait, or start a first run as above.
On a Bahriya Kubernetes Engine cluster at release 2.3 or later, the collector reads
the platform’s facts from the ConfigMap bke-platform through BKE’s
ClusterRole bke-platform-reader, bound in platform.namespace when the
chart is installed or upgraded there. Elsewhere the platform checks are
reported as not evaluated, never as failed.
Node facts
What the API cannot show, the kubelet’s settings, the modes of the files
Kubernetes keeps on each node and the node’s time service, is read on the
node itself. At each run the collector starts one short Job per node in the
namespace nodeCollector.namespace (the release’s namespace with -node
added, by default). Each Pod:
- runs as user 65534, with every capability dropped and no service account token;
- mounts only named host paths, each typed as a file or a directory and read-only, so a path that is absent is never created on the node;
- reads the kubelet’s configuration file and its flag files, and records the mode and owner of the files it cannot open;
- prints the node’s facts to its log, where the collector reads them, and sends nothing over the network.
The host paths mounted, each read-only, are /etc/kubernetes, the kubelet’s
configuration, flag file and certificates, /etc/default and
/etc/systemd; on a control-plane node, the etcd data directory and the API
server’s audit log directory, each taken from the API server’s or etcd’s own
volume.
The audit log directory. It is judged by its own check, the audit log permissions check, and not by the node file permissions check. The audit log check is judged when the API server writes its audit log to a directory on each control-plane node, held in a host-path volume; on a Bahriya Kubernetes Engine cluster, from the release that sets these modes, which the check’s reason names.
The node collector mounts the directory read-only and examines modes and
owners only; no code in it opens a file there. Because it runs as user
65534, it can read the audit log’s content only where the directory and a
file are already open to other users, which is the fault the check reports.
On a correctly set node it cannot list the files: a directory owned by root
and no more open than 700 then decides for them, since no user but root can
reach them, whatever their own modes. The check fails a wider directory, a
file wider than 600 or an owner other than root, and names the remedy:
chmod 0700 the directory, chmod 0600 each file in it and
chown root:root, as root on the node named. On BKE, upgrade.sh with
--commit sets them; a node moved to such a release by apply.sh alone
keeps its earlier modes until then.
The check passes only when the directory was examined on every control-plane node that runs an API server. Otherwise it is Not evaluated, and says why, beside each requirement that rests on it:
- the API server writes no audit log file, or writes it to its standard output;
- the audit log path is not absolute, or lies in no volume, or in a volume that is not a host directory;
- no node runs an API server Pod, as with a managed control plane;
kube-systemis not in the run’s scope;- the cluster runs a Bahriya Kubernetes Engine release that does not set these modes: the reason names the release to upgrade to.
The namespace is labelled Pod Security privileged, because every other
level refuses host paths. It holds nothing but these Jobs, the Convoy’s own
policies do not apply in it, and a NetworkPolicy allows no traffic in or out.
The collector may create, read and delete Jobs there and read their Pods and
logs; it cannot create a Pod directly. Facts read within
nodeCollector.maxAgeMinutes (60) by another regime’s run are used again.
The collector’s pull secrets must also exist in this namespace, since the node Jobs run the collector’s image.
The kubelet’s settings are those of its files: a change made after the
kubelet last started, and not yet in effect, is reported as though it were.
A node whose Job does not finish within nodeCollector.timeoutSeconds is
reported as not read, with the reason, and its checks as not evaluated.
nodeCollector.enabled=false removes the namespace and leaves every node
check not evaluated.
To pull the images from a mirror of your own, set
collector.image.repository and regimeImages.repositoryPrefix; each image
is still pulled by the digest the chart pins, so copy with
skopeo copy --all --preserve-digests.
In-cluster summary
The chart also creates trivy-operator’s ClusterComplianceReport
tersane-compliance-nist: trivy-operator’s checks grouped by the NIST SP
800-53 Rev. 5 controls the Convoy maps them to, refreshed every six hours
(summary.cron), for a quick view in the cluster:
kubectl get clustercompliancereport tersane-compliance-nist -o wide
It is indicative, not evidence. trivy-operator shows a check that has not
run as passed, and the summary covers only trivy-operator’s checks, keyed on
NIST rather than on any regime. The evidence for a regime is the report the
collector writes, which computes every result from the raw reports and
records a check with no result as not evaluated, and says what to fix. It is the same whichever
regimes the licence grants. summary.enabled=false removes it.
Checks the Convoy does not use, and why
The report names, for each of the Convoy’s checks, the trivy-operator checks it is evidenced from. Where a check of trivy-operator’s would give a wrong result for the clusters the Convoy serves, the Convoy does not use it, and the check is listed here with the reason.
AVD-KSV-0125: images from trusted registries
AVD-KSV-0125 reports an image as coming from an untrusted registry unless its
registry ends with an entry of a built-in list. That list names the registries
of three providers only: Microsoft Azure (azurecr.io), Amazon Elastic
Container Registry (the ecr.<region>.amazonaws.com domains) and Google
(gcr.io, us.gcr.io, eu.gcr.io and asia.gcr.io). All three providers are
based in the United States. Every other registry is reported as untrusted: a
registry in your own data centre, a registry in a national or in-country cloud,
and 1x.ax, from which Tersane’s own images are served.
For an organisation whose regulator or policy requires its systems and data to remain in the country, or inside its own perimeter, the result therefore reflects whether an image comes from one of those three providers, not whether it comes from a source the organisation trusts. Every registry the organisation runs itself, and every in-country registry other than those three providers’, is reported as untrusted.
The list cannot be replaced without losing the other checks. The check reads a
replacement list as rule data, which the trivy-operator version the
trivy-operator chart installs accepts only as a custom rule or library in its
policies ConfigMap; with any custom rule or
library present, it reports no built-in configuration check at all. Measured on
a test cluster, each workload’s report fell from 55 checks to none, with no
error logged.
The Convoy judges the same property against the registries you name instead.
You list your authorised registries in registries.allowed, each as the
beginning of an image reference (1x.ax/ by default).
policy.registries-allowed warns about an image from any other registry, or
refuses it once policies.actions.registries-allowed is Deny, and the report
evidences the result for every workload. trivy.image-sources continues to use AVD-KSV-0034 (images
from public registries) and AVD-KSV-0013 (images referenced by a moving tag).
Admission policies
Eleven ValidatingAdmissionPolicy objects, built into Kubernetes, so they
work whatever policy engine the cluster runs, or none. Each judges an object
as it is created or changed; Kyverno reports each one’s results for the
objects already in the cluster.
| Policy | Applies to | Requires |
|---|---|---|
registries-allowed |
Workloads | Images only from registries.allowed |
no-latest-tag |
Workloads | Every image by a fixed tag or digest |
no-privileged |
Workloads | No privileged container; no capability added but NET_BIND_SERVICE |
no-host-namespaces |
Workloads | No host network, process or IPC namespace |
no-host-ports |
Workloads | No host port |
no-host-path |
Workloads | No host path volume |
no-default-token |
Workloads | automountServiceAccountToken stated, true or false |
resource-limits |
Workloads | CPU and memory limits on every container |
namespace-environment |
Namespaces | The label tersane.bahriya.cloud/environment, one of namespaces.environments |
namespace-classification |
Namespaces | The label tersane.bahriya.cloud/data-classification, one of namespaces.classifications |
ingress-tls |
Ingresses | TLS for every host |
A workload is judged at its top-level object: a Deployment, StatefulSet, DaemonSet, Job, CronJob, or a Pod or ReplicaSet that nothing owns. The ReplicaSets and Pods a controller creates are not judged again, so a warning reaches whoever applied the workload and the report holds one result for it.
Each policy is Warn by default: the object is admitted, the person who
applied it sees a warning naming what failed, and the result is recorded.
Audit records the result without a warning. Move a policy to Deny once
nothing that should be admitted fails it, or to Off to remove it:
policies:
defaultAction: Warn
actions:
resource-limits: Deny
no-default-token: Off
No policy applies in the namespaces of policies.exemptNamespaces, nor in the
Convoy’s own. The defaults are the namespaces of a Bahriya Kubernetes Engine
cluster’s own components, among them netdata, and those of Kyverno and
trivy-operator; their components need what the policies forbid. The report
lists their failures under each check without counting them. The namespace
default is judged only while it holds a workload.
Kyverno policies
What the API server cannot decide itself is done with Kyverno.
| Policy | Kind | Establishes |
|---|---|---|
images-signed |
ImageValidatingPolicy |
Each container’s image is signed with cosign by the key that covers it; the collector’s report fails an image no key covers |
namespace-network-policy |
ValidatingPolicy |
Each application namespace holds at least one NetworkPolicy |
default-deny |
GeneratingPolicy |
A default-deny NetworkPolicy in each application namespace, when turned on |
Image signatures
Mamluk’s published signing keys are held by the chart, in files/signing/,
and verify every image matching signatures.mamluk.images (1x.ax/*). Each
entry of signatures.keys is a policy of its own,
tersane-compliance-images-signed-<name>, for the images matching its
patterns:
signatures:
keys:
- name: registry
images: ["registry.example.com/*"]
publicKeys:
- |
-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----
An image passes a policy when it is signed with any one of that policy’s keys,
so a key and its successor are listed together while a key is replaced. An
image matching no policy’s patterns is not checked for a signature;
registries-allowed decides where images may come from. Signatures in
cosign’s bundle format and in its earlier .sig format are both verified.
The policy changes nothing in the Pod: tags are not replaced with digests.
It follows policies.actions.images-signed like the built-in policies: Warn
admits the Pod and warns, naming each unsigned image, Deny refuses it, and
Off removes the policies.
Images in a private registry
Kyverno reads each signature from the registry that holds the image, with its
own pull secrets: those named in the kyverno chart’s existingImagePullSecrets,
1x-ax by default. For an image in any other registry that refuses anonymous
reads, Kyverno cannot read the signature, and the report lists the image as
failing with a detail such as:
evaluation error: failed to evaluate policy: GET https://registry.example.com/v2/payments/app/manifests/<tag>: UNAUTHORIZED: authentication required
Create a Secret of type kubernetes.io/dockerconfigjson for that registry in
Kyverno’s namespace, and name it in signatures.credentialSecrets. Write the
credential file first, as Creating the pull secret
does for 1x.ax, with your registry’s host and username in place of
registry.example.com and <username>; the shell asks for the password and does
not show it:
(
umask 077
printf 'Registry password: '; stty -echo; IFS= read -r TOKEN; stty echo; echo
printf '{"auths":{"registry.example.com":{"auth":"%s"}}}\n' \
"$(printf '%s:%s' '<username>' "$TOKEN" | base64 | tr -d '\n')" > registry.json
)
kubectl --namespace kyverno create secret generic example-registry \
--type=kubernetes.io/dockerconfigjson \
--from-file=.dockerconfigjson=registry.json
rm registry.json
signatures:
credentialSecrets: [example-registry]
Kyverno then reads signatures with these Secrets and with
signatures.kyvernoPullSecrets, and with nothing else: a policy’s credentials
replace Kyverno’s own rather than adding to them, so the chart lists Kyverno’s
pull secrets beside yours. Keep signatures.kyvernoPullSecrets the same as the
kyverno chart’s existingImagePullSecrets. A named Secret that does not exist
is skipped.
Where Mamluk’s images are mirrored into your own registry, point Mamluk’s key at
the mirror, name the mirror’s credential, and leave Kyverno’s pull secrets out
if no 1x-ax Secret exists:
signatures:
mamluk:
images: ["registry.example.com/mirror/*"]
credentialSecrets: [example-registry]
kyvernoPullSecrets: []
When Mamluk’s key is replaced, the new key is added to files/signing/
beside the old one, from bke/signing/, and the old one removed once every
published image carries the new signature.
NetworkPolicies
namespace-network-policy is evaluated by Kyverno’s background scan only,
because a namespace holds no NetworkPolicy at the moment it is created. Its
results are in Kyverno’s cluster policy reports, renewed when a namespace
changes and at each background scan. It is never refused at admission:
policies.defaultAction does not apply to it, and Deny named for it is
refused.
Namespaces in a service mesh. A service mesh pools connections, and a pooled connection opened before a NetworkPolicy change can outlive it. So the collector does not judge a meshed namespace’s segmentation from its NetworkPolicies where its traffic can be pooled; the check is then Not evaluated, and the report names each such namespace and why:
- a namespace in Istio ambient mode, whatever its Services declare;
- a namespace with Kuma sidecars where any Service port resolves, in Kuma’s
order, to
http,http2orgrpc: itsappProtocolwhere Kuma recognises it, otherwise the<port>.service.kuma.io/protocolannotation, otherwise TCP; - a namespace with Istio sidecars where any Service port does not declare TCP
explicitly (
appProtocol: tcp, or a name beginningtcp-), because Istio detects an undeclared port’s protocol itself; - a meshed namespace whose Services could not be read.
A namespace is meshed when a Pod in it carries a sidecar (kuma-sidecar,
istio-proxy), when a Pod’s own label or annotation enrols it, or when the
namespace’s label enrols it and not every Pod opts out. A label counts by its
value: for Kuma, enabled, true and yes enrol, and disabled, false
and no do not; a Pod’s own value overrides its namespace’s. Segmentation in
such a namespace rests on the mesh’s own traffic permissions, which the
collector does not read. A failure in another namespace still fails the
check. To read each Service’s port protocols, the collector may get and list
Services; of each it keeps only the ports’ names, their appProtocol and the
Kuma protocol annotation.
generate.defaultDenyNetworkPolicy.enabled has Kyverno write the
NetworkPolicy tersane-default-deny into each application namespace as it is
created. It selects every Pod and allows no traffic, in or out, except DNS
lookups to the cluster’s DNS service while allowDNS is true, so each
workload then needs NetworkPolicies of its own allowing what it uses. Kyverno
restores it when it is deleted or changed.
What a workload loses when it is written. On a cluster whose network enforces egress, as every Bahriya Kubernetes Engine cluster’s does, every Pod in the namespace at once stops reaching:
- other Pods and Services, in its namespace and elsewhere;
- the Kubernetes API server, so an operator, a controller or any workload that calls the API stops working;
- anything outside the cluster, including registries, databases and APIs it calls by address;
and nothing reaches it, ingress controllers included. Only DNS lookups remain,
while allowDNS is true. Before turning it on, give each application
namespace NetworkPolicies that allow its workloads’ traffic: for the API
server, an egress rule to the control-plane nodes’ addresses on port 6443.
existingNamespaces also writes it into the namespaces that exist when it is
turned on, which stops their traffic at once. It is off by default. Turning it
off again removes the policy and every NetworkPolicy it wrote.
On a Bahriya Kubernetes Engine cluster with the compliance setting. Where
BKE’s compliance.convoy setting is on, install the Convoy with this profile:
generate:
defaultDenyNetworkPolicy:
enabled: true
allowDNS: true
existingNamespaces: false
Every namespace created afterwards starts closed, so its workloads need NetworkPolicies of their own from the start, and the traffic of the namespaces that already exist is left as it is. A NetworkPolicy has no mode that only warns: it blocks traffic or does nothing, so the profile enforces from the first new namespace, and a namespace that existed before it, and has no NetworkPolicy of its own, keeps failing the network-policy check until its owners add one. On a new cluster there is no such namespace. Only Kubernetes NetworkPolicies count: a Calico staged policy, which records what it would do and enforces nothing, never passes the check.
The chart adds the permissions Kyverno’s controllers need for these, through ClusterRoles that Kyverno’s own roles aggregate.
Parameters
| Parameter | Description | Default |
|---|---|---|
regimes |
Regimes evidenced | [nca-ecc] |
scope.namespaces |
The namespaces to assess; empty assesses the whole cluster. See Scoped assessment | [] |
policies.defaultAction |
Warn, Audit or Deny for every policy |
Warn |
policies.actions |
Warn, Audit, Deny or Off for a policy, by name, including images-signed and namespace-network-policy |
{} |
policies.exemptNamespaces |
Namespaces no policy applies to | the platform’s, Kyverno’s and trivy-operator’s |
registries.allowed |
Registries images may come from, as reference prefixes | [1x.ax/] |
signatures.mamluk.images |
Images Mamluk’s keys verify, as patterns | [1x.ax/*] |
signatures.keys |
Keys of your own: name, images, publicKeys |
[] |
signatures.credentialSecrets |
Secrets in Kyverno’s namespace for reading signatures from other registries | [] |
signatures.kyvernoPullSecrets |
Kyverno’s own pull secrets, listed with credentialSecrets so that they are kept |
[1x-ax] |
generate.defaultDenyNetworkPolicy.enabled |
Write a default-deny NetworkPolicy into each application namespace | false |
generate.defaultDenyNetworkPolicy.allowDNS |
Allow DNS lookups in it | true |
generate.defaultDenyNetworkPolicy.existingNamespaces |
Also write it into existing namespaces | false |
namespaces.environments |
Values of the environment label | [production, staging, test, development] |
namespaces.classifications |
Values of the data classification label | [top-secret, secret, restricted, public] |
summary.enabled |
Create the NIST-keyed in-cluster summary (indicative) | true |
summary.cron |
When trivy-operator refreshes it | 0 */6 * * * |
collector.image.repository, .tag, .digest |
The collector image, pinned by digest | 1x.ax/tersane/convoy/compliance/core/collector |
collector.imagePullSecrets |
Pull secrets for the collector and regime images | [{name: 1x-ax}] |
collector.schedule |
When each regime’s collector runs | 17 2 * * * |
collector.activeDeadlineSeconds |
A run taking longer is stopped | 1800 |
collector.scanMaxAgeHours |
Scans older than this fail scan freshness | 48 |
collector.organisation, collector.cluster |
Shown on the report’s first page | "" |
collector.resources |
The collector’s requests and limits | 100m/256Mi, 1/2Gi |
webView.enabled |
Serve the runs, read-only, in the cluster | true |
webView.allowFrom |
NetworkPolicy peers that may reach the web view | [] (port-forward only) |
webView.url |
The web view’s address, shown in each email | "" |
email.mode |
none or smtp |
none |
email.smtp.host, .port, .tls |
Your SMTP server; starttls or implicit |
"", 587, starttls |
email.smtp.existingSecret, .usernameKey, .passwordKey |
The SMTP credential | "", username, password |
email.smtp.caSecret, .caKey |
A CA for the server’s certificate | "", ca.crt |
email.smtp.egressCIDRs |
The server’s addresses; required | [] |
email.from, email.to |
Sender and recipients | "", [] |
email.attach, email.maxAttachmentMB |
report.pdf, report.html; the size limit |
[report.pdf], 10 |
s3.enabled |
Upload each run | false |
s3.endpoint, .region, .bucket, .prefix, .pathStyle |
The store and where the objects go; the endpoint is required while S3 delivery is on | "", "", "", "", false |
s3.existingSecret, .accessKeyIdKey, .secretAccessKeyKey |
The access key | "", accessKeyId, secretAccessKey |
s3.caSecret, .caKey |
A CA for the store’s certificate | "", ca.crt |
s3.egressCIDRs |
The store’s addresses; required | [] |
apiServer.cidrs, .port |
The API server’s addresses, where the chart cannot read them | [], 6443 |
nodeCollector.enabled |
Read each node’s facts | true |
nodeCollector.namespace |
The node Jobs’ namespace, labelled privileged | "" (<namespace>-node) |
nodeCollector.maxAgeMinutes |
Node facts this recent are used again | 60 |
nodeCollector.timeoutSeconds |
How long a run waits for the nodes | 300 |
evidence.retentionDays |
Days of runs kept | 400 |
evidence.size, evidence.storageClass |
Each regime’s volume, sized by the table in values.yaml |
10Gi, the default class |
evidence.signingKey.existingSecret, .key |
A Secret with the PEM key that signs each manifest | "", key.pem |
regimeImages.repositoryPrefix |
A mirror holding the regime images | "" (1x.ax) |
platform.namespace |
Where BKE records its platform facts | bahriya-system |
platform.createBinding |
Bind the collector there whether or not the chart can see the namespace; for helm template on BKE |
false |
longhorn.createBinding |
Let the collector read Longhorn’s backup targets, recurring jobs and volumes in longhorn-system whether or not the chart can see the namespace; for helm template |
false |
fluentBit.createBinding |
Let the collector read the ConfigMaps in fluentBit.configMaps in logging whether or not the chart can see the namespace; for helm template |
false |
fluentBit.configMaps |
The ConfigMaps in logging the collector may read: the one Fluent Bit’s DaemonSet mounts, named after its Helm release |
[bke-fluent-bit, fluent-bit] |
fluentBit.metricsPort |
The port of Fluent Bit’s HTTP server, from which the collector reads each Pod’s delivery counters; the collector’s NetworkPolicy admits this port alone, on Fluent Bit’s Pods in logging |
2020 |
prerequisites.check |
Refuse to install without Kyverno and trivy-operator | true |