Key Manager service for secrets and encryption keys

The Key Manager service (barbican) centrally manages certificates, keys, and passwords so that you can protect sensitive data used by OpenStack services.

The Key Manager Operator is installed and active by default. You can use the Key Manager API and command line to manage secrets. The RHOSO Key Manager service integrates with the Block Storage (cinder), Networking (neutron), and Compute (nova) services.

Secrets such as certificates, API keys, and passwords, are stored in the Key Manager service database or directly in a secure storage system. In RHOSO 18.0 the simple crypto plugin is available to encrypt secrets.

OpenStack Key Manager supports the following use cases:

Symmetric encryption keys
- Block Storage (cinder) volume encryption
- Object Storage (swift) encryption
- Ephemeral disk encryption
Asymmetric keys and certificates
- Image service (glance) image signing
- Image service (glance) image verification

View secrets in the Key Manager service
Copy link

View secrets in the Key Manager service to review their properties, such as URI, name, type, and expiration, so that you can confirm they are configured correctly.

About this task
Copy link

Note

You cannot view the payload content of the secrets.

Procedure
Copy link

View the list of secrets:

$ openstack secret list

For example:

+------------------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| Secret href                                                                        | Name | Created                   | Status | Content types                             | Algorithm | Bit length | Secret type | Mode | Expiration |
+------------------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| https://192.168.123.169:9311/v1/secrets/24845e6d-64a5-4071-ba99-0fdd1046172e       | None | 2018-01-22T02:23:15+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
+------------------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+

Store a secret in the Key Manager service
Copy link

Store a secret in the Key Manager service so that your OpenStack services can securely retrieve and use sensitive data such as certificates and keys.

About this task
Copy link

You must specify the name of the secret. You can optionally specify its payload to attach the actual content to the secret metadata. If you omit the payload, you can add it later.

Note

You cannot change the payload of a secret, you can only delete it. If you create a secret without specifying a payload, then you can later add a payload to it by using the openstack secret update command.

Procedure
Copy link

Create a secret:

$ openstack secret store --name testSecret --payload 'TestPayload'

For example:

+---------------+------------------------------------------------------------------------------------+
| Field         | Value                                                                              |
+---------------+------------------------------------------------------------------------------------+
| Secret href   | https://192.168.123.163:9311/v1/secrets/ecc7b2a4-f0b0-47ba-b451-0f7d42bc1746       |
| Name          | testSecret                                                                         |
| Created       | None                                                                               |
| Status        | None                                                                               |
| Content types | None                                                                               |
| Algorithm     | aes                                                                                |
| Bit length    | 256                                                                                |
| Secret type   | opaque                                                                             |
| Mode          | cbc                                                                                |
| Expiration    | None                                                                               |
+---------------+------------------------------------------------------------------------------------+

Add a payload to a secret
Copy link

Add a payload to an existing secret in the Key Manager service so that services can retrieve and use the stored sensitive data, such as a certificate or key.

About this task
Copy link

Note

You cannot change the payload of a secret, you can only delete it.

If you create a secret without specifying a payload, then you can add a payload to it by using the openstack secret update command.

Procedure
Copy link

Add a payload to a secret that does not contain a payload yet:

$ openstack secret update https://192.168.123.163:9311/v1/secrets/ca34a264-fd09-44a1-8856-c6e7116c3b16 'TestPayload'

Remove a secret from the Key Manager service
Copy link

Delete a secret from the Key Manager service (barbican) to permanently remove obsolete or compromised sensitive data and prevent unauthorized access.

Procedure
Copy link

You must specify the secret URI when deleting a secret:

Tip

You can use this command openstack secret list to obtain the URI of the secret.

$ openstack secret delete https://192.168.123.163:9311/v1/secrets/ecc7b2a4-f0b0-47ba-b451-0f7d42bc1746

Generate a symmetric encryption key
Copy link

Generate a symmetric encryption key in the Key Manager service (barbican) so that you can enable encryption for services such as Compute disk encryption and Object Storage encryption.

Procedure
Copy link

Generate a new 256-bit key using order create and store it in the Key Manager service.

$ openstack secret order create --name swift_key --algorithm aes --mode ctr --bit-length 256 --payload-content-type=application/octet-stream key

For example:

+----------------+-----------------------------------------------------------------------------------+
| Field          | Value                                                                             |
+----------------+-----------------------------------------------------------------------------------+
| Order href     | https://192.168.123.173:9311/v1/orders/043383fe-d504-42cf-a9b1-bc328d0b4832       |
| Type           | Key                                                                               |
| Container href | N/A                                                                               |
| Secret href    | None                                                                              |
| Created        | None                                                                              |
| Status         | None                                                                              |
| Error code     | None                                                                              |
| Error message  | None                                                                              |
+----------------+-----------------------------------------------------------------------------------+

You can also use the --mode option to configure generated keys to use a particular mode, such as ctr or cbc. For more information, see NIST SP 800-38A.

View the details of the order to identify the location of the generated key, shown here as the Secret href value:

Example command:

$ openstack secret order get https://192.168.123.173:9311/v1/orders/043383fe-d504-42cf-a9b1-bc328d0b4832

For example:

+----------------+------------------------------------------------------------------------------------+
| Field          | Value                                                                              |
+----------------+------------------------------------------------------------------------------------+
| Order href     | https://192.168.123.173:9311/v1/orders/043383fe-d504-42cf-a9b1-bc328d0b4832        |
| Type           | Key                                                                                |
| Container href | N/A                                                                                |
| Secret href    | https://192.168.123.173:9311/v1/secrets/efcfec49-b9a3-4425-a9b6-5ba69cb18719       |
| Created        | 2018-01-24T04:24:33+00:00                                                          |
| Status         | ACTIVE                                                                             |
| Error code     | None                                                                               |
| Error message  | None                                                                               |
+----------------+------------------------------------------------------------------------------------+

Retrieve the details of the secret:

Example command:

$ openstack secret get https://192.168.123.173:9311/v1/secrets/efcfec49-b9a3-4425-a9b6-5ba69cb18719

For example:

+---------------+------------------------------------------------------------------------------------+
| Field         | Value                                                                              |
+---------------+------------------------------------------------------------------------------------+
| Secret href   | https://192.168.123.173:9311/v1/secrets/efcfec49-b9a3-4425-a9b6-5ba69cb18719       |
| Name          | swift_key                                                                          |
| Created       | 2018-01-24T04:24:33+00:00                                                          |
| Status        | ACTIVE                                                                             |
| Content types | {u'default': u'application/octet-stream'}                                          |
| Algorithm     | aes                                                                                |
| Bit length    | 256                                                                                |
| Secret type   | symmetric                                                                          |
| Mode          | ctr                                                                                |
| Expiration    | None                                                                               |
+---------------+------------------------------------------------------------------------------------+

Encrypt Block Storage volumes with the Key Manager service
Copy link

Secure your storage resources with volume encryption with the Key Manager service (barbican). Volume contents remain unreadable to unauthorized users even if the physical storage is compromised.

About this task
Copy link

You can use the Key Manager service to manage your Block Storage service (cinder) encryption keys. This configuration uses LUKS to encrypt the disks attached to your instances, including boot disks.

Block Storage volume key management is transparent to the user; when you create a new volume with luks as the encryption type, the Block Storage service generates a symmetric key secret for the volume and stores it in the Key Manager service. When booting the instance or attaching an encrypted volume, the Compute service (nova) retrieves the key from the Key Manager service and stores the secret locally as a libvirt secret on the Compute node.

Note

When creating a Block Storage volume that uses encryption, you must use the luks encryption provider, the aes-xts-plain64 encryption cipher, and use a key size of 256.

Procedure
Copy link

Create a Block Storage volume type that uses encryption.

$ openstack volume type create \
--encryption-provider  nova.volume.encryptors.luks.LuksEncryptor \
--encryption-cipher aes-xts-plain64 \
--encryption-key-size 256 \
--encryption-control-location front-end LuksEncryptor-256

+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field       | Value                                                                                                                                                                              |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| description | None                                                                                                                                                                               |
| encryption  | cipher='aes-xts-plain64', control_location='front-end', encryption_id='9df604d0-8584-4ce8-b450-e13e6316c4d3', key_size='256', provider='nova.volume.encryptors.luks.LuksEncryptor' |
| id          | 78898a82-8f4c-44b2-a460-40a5da9e4d59                                                                                                                                               |
| is_public   | True                                                                                                                                                                               |
| name        | LuksEncryptor-256                                                                                                                                                         |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Create a new Block Storage volume and specify that it uses the LuksEncryptor-256 volume type:

$ openstack volume create --size 1 --type LuksEncryptor-256 'Encrypted-Test-Volume'

+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| attachments         | []                                   |
| availability_zone   | nova                                 |
| bootable            | false                                |
| consistencygroup_id | None                                 |
| created_at          | 2018-01-22T00:19:06.000000           |
| description         | None                                 |
| encrypted           | True                                 |
| id                  | a361fd0b-882a-46cc-a669-c633630b5c93 |
| migration_status    | None                                 |
| multiattach         | False                                |
| name                | Encrypted-Test-Volume                |
| properties          |                                      |
| replication_status  | None                                 |
| size                | 1                                    |
| snapshot_id         | None                                 |
| source_volid        | None                                 |
| status              | creating                             |
| type                | LuksEncryptor-256           |
| updated_at          | None                                 |
| user_id             | 0e73cb3111614365a144e7f8f1a972af     |
+---------------------+--------------------------------------+

The resulting secret is automatically uploaded to the Key Manager service back end.

Ensure that you can see the Key Manager service secret UUID. This value is displayed in the encryption_key_id field. You must use the --os-volume-api-version 3.64 parameter with the Block Storage service cinder CLI to display the encryption_key_id value. There is no equivalent openstack CLI command.

$ cinder --os-volume-api-version 3.64 volume show Encrypted-Test-Volume

+------------------------------+-------------------------------------+
|Property                      |Value                                |
+------------------------------+-------------------------------------+
|attached_servers              |[]                                   |
|attachment_ids                |[]                                   |
|availability_zone             |nova                                 |
|bootable                      |false                                |
|cluster_name                  |None                                 |
|consistencygroup_id           |None                                 |
|created_at                    |2022-07-28T17:35:26.000000           |
|description                   |None                                 |
|encrypted                     |True                                 |
|encryption_key_id             |0944b8a8-de09-4413-b2ed-38f6c4591dd4 |
|group_id                      |None                                 |
|id                            |a0b51b97-0392-460a-abfa-093022a120f3 |
|metadata                      |                                     |
|migration_status              |None                                 |
|multiattach                   |False                                |
|name                          |vol                                  |
|os-vol-host-attr:host         |hostgroup@tripleo_iscsi#tripleo_iscsi|
|os-vol-mig-status-attr:migstat|None                                 |
|os-vol-mig-status-attr:name_id|None                                 |
|os-vol-tenant-attr:tenant_id  |a2071ece39b3440aa82395ff7707996f     |
|provider_id                   |None                                 |
|replication_status            |None                                 |
|service_uuid                  |471f0805-072e-4256-b447-c7dd10ceb807 |
|shared_targets                |False                                |
|size                          |1                                    |
|snapshot_id                   |None                                 |
|source_volid                  |None                                 |
|status                        |available                            |
|updated_at                    |2022-07-28T17:35:26.000000           |
|user_id                       |ba311b5c2b8e438c951d1137333669d4     |
|volume_type                   |LUKS                                 |
|volume_type_id                |cc188ace-f73d-4af5-bf5a-d70ccc5a401c |
+------------------------------+-------------------------------------+

Use the Key Manager service to confirm that the disk encryption key is present. In this example, the timestamp matches the LUKS volume creation time:

$ openstack secret list

+------------------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| Secret href                                                                        | Name | Created                   | Status | Content types                             | Algorithm | Bit length | Secret type | Mode | Expiration |
+------------------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| https://192.168.123.169:9311/v1/secrets/0944b8a8-de09-4413-b2ed-38f6c4591dd4 | None | 2018-01-22T02:23:15+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
+------------------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+

Attach the new Block Storage volume to an existing instance:

$ openstack server add volume testInstance Encrypted-Test-Volume

Sign Image service images
Copy link

Validate Image service (glance) images to ensure data integrity. Verify signatures against trusted keys to prevent the deployment of compromised images that have been modified by unauthorized users.

About this task
Copy link

Sign Image service (glance) images before uploading them so that signature verification prevents the deployment of compromised or tampered images. Use the openssl command to sign an image with a key that is stored in the Key Manager service (barbican), and then upload the image with the accompanying signature properties. The signature is verified before each use, and the instance build process fails if the signature does not match.

Limitations

Signature verification is based on openSSL _rsa_sig_verify(), which currently raises an InvalidSignature exception in the Image service if the key length is not 1024. This issue occurs because of a regression in OpenSSL 3.0.7. Signature verification works correctly with key length 2048 and OpenSSL 3.0.2.
OpenSSL does not support the SHA256 algorithm. You must use SHA512 or greater to create a signed image.

Procedure
Copy link

Use the openssl command to create the keys and the certificate:

Create the private key:

$ openssl genrsa -out private_key.pem 1024

Create the public key:

$ openssl rsa -pubout -in private_key.pem -out public_key.pem

Create the certificate:

$ openssl req -new -key private_key.pem -out cert_request.csr

When prompted, enter details to include in your certificate request:

$ openssl x509 -req -days 14 -in cert_request.csr -signkey private_key.pem -out new_cert.crt

Upload the certificate to the Key Manager service, where it is stored in the secret store:

$ openstack secret store --name test --algorithm RSA --secret-type certificate \
  --payload-content-type "application/octet-stream" \
  --payload-content-encoding base64 \
  --payload "$(base64 new_cert.crt)"

For example:

+---------------+-----------------------------------------------------------------------+
| Field         | Value                                                                 |
+---------------+-----------------------------------------------------------------------+
| Secret href   | http://127.0.0.1:9311/v1/secrets/cd7cc675-e573-419c-8fff-33a72734a243 |
+---------------+-----------------------------------------------------------------------+

Note

Record the certificate’s UUID for use in a later step. In this example, the UUID is cd7cc675-e573-419c-8fff-33a72734a243.

Retrieve an image and create a signature:
- Retrieve an image:
  $ echo <This is my image> > <myimage>
- Use private_key.pem to sign the image and generate the .signature file. For example:
  $ openssl dgst -sha512 -sign private_key.pem -sigopt rsa_padding_mode:pss -out myimage.signature myimage
- Convert the .signature file to Base64 format:
  $ base64 -w 0 myimage.signature > myimage.signature.b64
- Load the Base64 value into a variable to use when you upload the image:
  $ image_signature=$(cat myimage.signature.b64)

Upload an image with valid signature properties to the Image service:

$ openstack image-create \
  --name <my_signed_image> \
  --container-format bare \
  --disk-format qcow2 \
  --property img_signature="$image_signature" \
  --property img_signature_certificate_uuid="$cert_uuid" \
  --property img_signature_hash_method='SHA-512' \
  --property img_signature_key_type='RSA-PSS' < myimage

Validate snapshots
Copy link

Validate snapshots to verify their integrity before use.

About this task
Copy link

Snapshots are saved as Image service (glance) images. If you configure the Compute service to check for signed images, then snapshots must be signed, even if they were created from an instance with a signed image.

Procedure
Copy link

Download the snapshot from the Image service:
```
$ openstack image save --file <local_file_name> <snapshot_image_name>
```
- Replace <local_file_name> with the file name of the downloaded image.
- Replace <snapshot_image_name> with the name of your snapshot.
Generate a signature to validate the snapshot. Use the same process that you use to generate a signature to validate an image. For more information, see Signing Image service (glance) images.

Update the snapshot image properties:

$ openstack image set \
  --property img_signature="$image_signature" \
  --property img_signature_certificate_uuid="<cd7cc675-e573-419c-8fff-33a72734a243>" \
  --property img_signature_hash_method="SHA-512" \
  --property img_signature_key_type="RSA-PSS" \
  <snapshot_image_id>

Replace <cd7cc675-e573-419c-8fff-33a72734a243> with the UUID of the certificate for the image.
Replace <snapshot_image_id> with the ID of your snapshot.

Optional: Remove the downloaded Image service image from the filesystem:
```
$ rm <local_file_name>
```

Back up simple crypto encryption keys
Copy link

Back up the Key Manager service (barbican) encryption keys so that you can restore access to encrypted volumes during disaster recovery.

Before you begin
Copy link

You have a security-hardened location for the BarbicanSimpleCryptoKEK key and Key Manager database.

About this task
Copy link

To back up simple crypto encryption keys, you must back up the Key Manager database and the BarbicanSimpleCryptoKek key that encrypts and decrypts values from the database. Store the key and Key Manager database in a security-hardened location.

Procedure
Copy link

Retrieve the base64 decoded BarbicanSimpleCryptoKEK value from osp-secret and save it to a file called kek.txt:

$ oc get secret osp-secret -o yaml | grep BarbicanSimpleCryptoKEK | awk '{print $2}' | base64 -d > kek.txt

Retrieve the Key Manager database root password:

$ DB_PASS=$(oc get secret osp-secret -o yaml | grep DbRootPassword | awk '{print $2}' | base64 -d)

Use the Key Manager database root password to back up the 'barbican' database:

$ oc exec -it openstack-galera-0 -- mysqldump -u root -p"${DB_PASS}" barbican > barbican_db_backup.sql

Check that the database backup is stored in your current working directory:

$ ll
total 36
-rw-rw-r--. 1 tripleo-admin tripleo-admin 36715 Jun 19 18:31 barbican_db_backup.sql

Back up the barbican_db_backup.sql and kek.txt files to the security hardened location.

Restore simple crypto encryption keys
Copy link

Restore the Key Manager service (barbican) encryption keys from a backup so that you can regain access to encrypted volumes after a system failure.

About this task
Copy link

Restore the Key Manager database and the BarbicanSimpleCryptoKek password from your backup to restore simple crypto encryption keys.

Procedure
Copy link

Encode your BarbicanSimpleCryptoKek key using base64 and write the resulting value into osp-secret:
```
$ echo -n <barbican_key> | base64
YmFyYmljYW5fc2ltcGxlX2N5cHRvX2tlawo=
```
Replace <barbican_key> with the BarbicanSimpleCryptoKek key that you want to restore.
Open the osp-secret secret object for editing:
```
$ oc edit secret osp-secret
```

Add the value from step 1 and save your changes:

....
  BarbicanDatabasePassword: cGFzc3dvcmQK
  BarbicanPassword: cGFzc3dvcmQK
  BarbicanSimpleCryptoKEK: YmFyYmljYW5fc2ltcGxlX2N5cHRvX2tlawo=
  CeilometerPassword: cGFzc3dvcmQK
  CinderDatabasePassword: cGFzc3dvcmQK
  CinderPassword: cGFzc3dvcmQK
  DatabasePassword: cGFzc3dvcmQK
....

Retrieve the Key Manager database root password:

DB_PASS=$(oc get secret osp-secret -o yaml | grep DbRootPassword | awk '{print $2}' | base64 -d)

Restore the backup file to the Key Manager barbican database:

$ oc exec openstack-galera-0 --  mysql -u root -p"${DB_PASS}" barbican < <sql_backup>

Replace <sql_backup> with the name of your mysql database backup file.
For example:
```
Defaulted container "galera" out of: galera, mysql-bootstrap (init)
```

Results
Copy link

You can use the openstackclient pod to verify that the test secrets were restored successfully:

$ oc exec -n openstack -t openstackclient -- openstack secret list
+------------------------------------------------------------------------+------------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| Secret href                                                        	| Name   	| Created               	| Status | Content types                         	| Algorithm | Bit length | Secret type | Mode | Expiration |
+------------------------------------------------------------------------+------------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| http://10.0.0.104:9311/v1/secrets/93f62cfd-e008-401f-be74-bf057c88b04a | testSecret | 2018-06-19T18:25:25+00:00 | ACTIVE | {u'default': u'text/plain'}           	| aes   	|    	256 | opaque  	| cbc  | None   	|
| http://10.0.0.104:9311/v1/secrets/f664b5cf-5221-47e5-9887-608972a5fefb | swift_key  | 2018-06-19T18:24:40+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes   	|    	256 | symmetric   | ctr  | None   	|
+------------------------------------------------------------------------+------------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+

On this page

On this page

Key Manager service for secrets and encryption keys

View secrets in the Key Manager serviceCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

Store a secret in the Key Manager serviceCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

Add a payload to a secretCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

Remove a secret from the Key Manager serviceCopy linkLink copied!

ProcedureCopy linkLink copied!

Generate a symmetric encryption keyCopy linkLink copied!

ProcedureCopy linkLink copied!

Encrypt Block Storage volumes with the Key Manager serviceCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

Sign Image service imagesCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

Validate snapshotsCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

Back up simple crypto encryption keysCopy linkLink copied!

Before you beginCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

Restore simple crypto encryption keysCopy linkLink copied!

About this taskCopy linkLink copied!

ProcedureCopy linkLink copied!

ResultsCopy linkLink copied!

Learn

Try, buy, & sell

Communities

About Red Hat

Making open source more inclusive

About Red Hat Documentation

Theme

Red Hat legal and privacy links

Red Hat legal and privacy links

View secrets in the Key Manager service
Copy link

About this task
Copy link

Procedure
Copy link

Store a secret in the Key Manager service
Copy link

About this task
Copy link

Procedure
Copy link

Add a payload to a secret
Copy link

About this task
Copy link

Procedure
Copy link

Remove a secret from the Key Manager service
Copy link

Procedure
Copy link

Generate a symmetric encryption key
Copy link

Procedure
Copy link

Encrypt Block Storage volumes with the Key Manager service
Copy link

About this task
Copy link

Procedure
Copy link

Sign Image service images
Copy link

About this task
Copy link

Procedure
Copy link

Validate snapshots
Copy link

About this task
Copy link

Procedure
Copy link

Back up simple crypto encryption keys
Copy link

Before you begin
Copy link

About this task
Copy link

Procedure
Copy link

Restore simple crypto encryption keys
Copy link

About this task
Copy link

Procedure
Copy link

Results
Copy link