Tang server encryption key management - Network-Bound Disk Encryption (NBDE) | Security and compliance

Backing up keys for a Tang server
Recovering keys for a Tang server
Rekeying Tang servers
Deleting old Tang server keys

The cryptographic mechanism to recreate the encryption key is based on the blinded key stored on the node and the private key of the involved Tang servers. To protect against the possibility of an attacker who has obtained both the Tang server private key and the node’s encrypted disk, periodic rekeying is advisable.

You must perform the rekeying operation for every node before you can delete the old key from the Tang server. The following sections provide procedures for rekeying and deleting old keys.

Backing up keys for a Tang server

The Tang server uses /usr/libexec/tangd-keygen to generate new keys and stores them in the /var/db/tang directory by default. To recover the Tang server in the event of a failure, back up this directory. The keys are sensitive and because they are able to perform the boot disk decryption of all hosts that have used them, the keys must be protected accordingly.

Procedure

Copy the backup key from the /var/db/tang directory to the temp directory from which you can restore the key.

Recovering keys for a Tang server

You can recover the keys for a Tang server by accessing the keys from a backup.

Procedure

Restore the key from your backup folder to the /var/db/tang/ directory.

When the Tang server starts up, it advertises and uses these restored keys.

Rekeying Tang servers

This procedure uses a set of three Tang servers, each with unique keys, as an example.

Using redundant Tang servers reduces the chances of nodes failing to boot automatically.

Rekeying a Tang server, and all associated NBDE-encrypted nodes, is a three-step procedure.

Prerequisites

A working Network-Bound Disk Encryption (NBDE) installation on one or more nodes.

Procedure

Generate a new Tang server key.
Rekey all NBDE-encrypted nodes so they use the new key.
Delete the old Tang server key.

Deleting the old key before all NBDE-encrypted nodes have completed their rekeying causes those nodes to become overly dependent on any other configured Tang servers.

Figure 1. Example workflow for rekeying a Tang server

Generating a new Tang server key

Prerequisites

A root shell on the Linux machine running the Tang server.

To facilitate verification of the Tang server key rotation, encrypt a small test file with the old key:

# echo plaintext | clevis encrypt tang '{"url":"http://localhost:7500”}' -y >/tmp/encrypted.oldkey

Verify that the encryption succeeded and the file can be decrypted to produce the same string plaintext:
```
# clevis decrypt </tmp/encrypted.oldkey
```

Procedure

Locate and access the directory that stores the Tang server key. This is usually the /var/db/tang directory. Check the currently advertised key thumbprint:
```
# tang-show-keys 7500
```
Example output
```
36AHjNH3NZDSnlONLz1-V4ie6t8
```
Enter the Tang server key directory:
```
# cd /var/db/tang/
```
List the current Tang server keys:
```
# ls -A1
```
Example output
```
36AHjNH3NZDSnlONLz1-V4ie6t8.jwk
gJZiNPMLRBnyo_ZKfK4_5SrnHYo.jwk
```
During normal Tang server operations, there are two .jwk files in this directory: one for signing and verification, and another for key derivation.
Disable advertisement of the old keys:
```
# for key in *.jwk; do \
  mv -- "$key" ".$key"; \
done
```
New clients setting up Network-Bound Disk Encryption (NBDE) or requesting keys will no longer see the old keys. Existing clients can still access and use the old keys until they are deleted. The Tang server reads but does not advertise keys stored in UNIX hidden files, which start with the . character.

Generate a new key:

# /usr/libexec/tangd-keygen /var/db/tang

List the current Tang server keys to verify the old keys are no longer advertised, as they are now hidden files, and new keys are present:
```
# ls -A1
```
Example output
```
.36AHjNH3NZDSnlONLz1-V4ie6t8.jwk
.gJZiNPMLRBnyo_ZKfK4_5SrnHYo.jwk
Bp8XjITceWSN_7XFfW7WfJDTomE.jwk
WOjQYkyK7DxY_T5pMncMO5w0f6E.jwk
```
Tang automatically advertises the new keys.

More recent Tang server installations include a helper /usr/libexec/tangd-rotate-keys directory that takes care of disabling advertisement and generating the new keys simultaneously.
If you are running multiple Tang servers behind a load balancer that share the same key material, ensure the changes made here are properly synchronized across the entire set of servers before proceeding.

Verification

Verify that the Tang server is advertising the new key, and not advertising the old key:
```
# tang-show-keys 7500
```
Example output
```
WOjQYkyK7DxY_T5pMncMO5w0f6E
```
Verify that the old key, while not advertised, is still available to decryption requests:
```
# clevis decrypt </tmp/encrypted.oldkey
```

Rekeying all NBDE nodes

You can rekey all of the nodes on a remote cluster by using a DaemonSet object without incurring any downtime to the remote cluster.

If a node loses power during the rekeying, it is possible that it might become unbootable, and must be redeployed via Red Hat Advanced Cluster Management (RHACM) or a GitOps pipeline.

Prerequisites

cluster-admin access to all clust