Bahriya Kubernetes Engine 2.5.0 is released. It is a minor release: it changes each node’s boot configuration and the kubelet’s behaviour, with one new config.yaml key. No Kubernetes, containerd, Calico or component version moves. Exact versions are in the release notes in the Mamluk Console.
No node is drained for this release. Each node is upgraded in turn, which restarts its kubelet, so plan it as a maintenance window.
Kernel settings and modules at every boot
Every node now holds BKE’s kernel settings, and loads the two kernel modules Kubernetes and Longhorn need, at every boot. Until this release the modules were loaded only when the node was installed, so after a reboot they, and the settings that depend on them, could be absent until something loaded them.
The upgrade reads every setting back, and stops a node before its kubelet changes if a setting did not take, or if a file of your own would replace one of BKE’s at the next boot. Run it without --commit on every node days before the window: it lists what it will change, and names any such file for you to correct.
The kubelet does not start with the wrong kernel settings
The kubelet now runs with protectKernelDefaults on. Until this release it corrected six kernel settings itself each time it started, so a node whose settings had changed was corrected without anything saying so. With the protection on, the kubelet does not start on such a node. BKE writes the same six values first and reads them back before it turns the protection on.
A kubelet that refuses to start can go on showing Ready, so upgrade.sh watches it after restarting it, and check.sh fails on a kubelet that is restarting. To keep the earlier behaviour, set kubelet.protectKernelDefaults: false in config.yaml and keep it there.
The audit log directory
The API audit log directory on each control-plane node is now closed to other local users, as each audit file already was. check.sh warns about a directory or file others can read.
Upgrading
Clusters upgrade from 2.4.x. A minor release cannot be skipped, and check.sh refuses an attempt to skip one.
The order is the usual one: upgrade.sh on the first control-plane node, then on each other node in turn, then apply.sh for the components. As with every BKE upgrade, apply.sh first reports each change against your own values files and changes nothing until you run it with --commit. The release notes list what has not yet been tested on servers, including clusters with more than one control-plane node. The full sequence is in the documentation.