Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PG-1457 Key management funcs renaming #126

Open
wants to merge 17 commits into
base: release-17.4
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ci_scripts/backup/pg_basebackup_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,8 @@ setup_tde_heap(){
sudo -u "$PG_USER" psql -p $PG_PORT -c "DROP DATABASE IF EXISTS $DB_NAME;"
sudo -u "$PG_USER" psql -p $PG_PORT -c "CREATE DATABASE $DB_NAME;"
sudo -u "$PG_USER" psql -d "$DB_NAME" -p "$PG_PORT" -c "CREATE EXTENSION IF NOT EXISTS pg_tde;"
sudo -u "$PG_USER" psql -d "$DB_NAME" -p "$PG_PORT" -c "SELECT pg_tde_add_key_provider_file('file-vault','$KEYLOCATION');"
sudo -u "$PG_USER" psql -d "$DB_NAME" -p "$PG_PORT" -c "SELECT pg_tde_set_principal_key('test-db-master-key','file-vault');"
sudo -u "$PG_USER" psql -d "$DB_NAME" -p "$PG_PORT" -c "SELECT pg_tde_add_database_key_provider_file('file-vault','$KEYLOCATION');"
sudo -u "$PG_USER" psql -d "$DB_NAME" -p "$PG_PORT" -c "SELECT pg_tde_set_principal_key_using_database_key_provider('test-db-master-key','file-vault');"
sudo -u "$PG_USER" psql -p $PG_PORT -c "ALTER DATABASE $DB_NAME SET default_table_access_method='tde_heap';"
sudo -u "$PG_USER" psql -p $PG_PORT -c "SELECT pg_reload_conf();"
}
Expand Down
4 changes: 2 additions & 2 deletions ci_scripts/tde_setup.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
CREATE SCHEMA IF NOT EXISTS tde;
CREATE EXTENSION IF NOT EXISTS pg_tde SCHEMA tde;
SELECT pg_tde_add_key_provider_file('reg_file-vault', '/tmp/pg_tde_test_keyring.per');
SELECT pg_tde_set_principal_key('test-db-principal-key', 'reg_file-vault');
SELECT pg_tde_add_database_key_provider_file('reg_file-vault', '/tmp/pg_tde_test_keyring.per');
SELECT pg_tde_set_principal_key_using_database_key_provider('test-db-principal-key', 'reg_file-vault');
2 changes: 1 addition & 1 deletion ci_scripts/tde_setup_global.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ CREATE SCHEMA tde;
CREATE EXTENSION IF NOT EXISTS pg_tde SCHEMA tde;

SELECT tde.pg_tde_add_global_key_provider_file('reg_file-global', '/tmp/pg_tde_test_keyring.per');
SELECT tde.pg_tde_set_server_principal_key('global-principal-key', 'reg_file-global');
SELECT tde.pg_tde_set_server_principal_key_using_global_key_provider('server-principal-key', 'reg_file-global');
ALTER SYSTEM SET pg_tde.wal_encrypt = on;
ALTER SYSTEM SET default_table_access_method = 'tde_heap';
ALTER SYSTEM SET search_path = "$user",public,tde;
Expand Down
12 changes: 6 additions & 6 deletions contrib/pg_tde/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,25 +119,25 @@ _See [Make Builds for Developers](https://github.com/percona/pg_tde/wiki/Make-bu

```sql
-- For Vault-V2 key provider
-- pg_tde_add_key_provider_vault_v2(provider_name, vault_token, vault_url, vault_mount_path, vault_ca_path)
SELECT pg_tde_add_key_provider_vault_v2(
-- pg_tde_add_database_key_provider_vault_v2(provider_name, vault_token, vault_url, vault_mount_path, vault_ca_path)
SELECT pg_tde_add_database_key_provider_vault_v2(
'vault-provider',
json_object( 'type' VALUE 'remote', 'url' VALUE 'http://localhost:8888/token' ),
json_object( 'type' VALUE 'remote', 'url' VALUE 'http://localhost:8888/url' ),
to_json('secret'::text), NULL);

-- For File key provider
-- pg_tde_add_key_provider_file(provider_name, file_path);
SELECT pg_tde_add_key_provider_file('file','/tmp/pgkeyring');
-- pg_tde_add_database_key_provider_file(provider_name, file_path);
SELECT pg_tde_add_database_key_provider_file('file','/tmp/pgkeyring');
```

**Note: The `File` provided is intended for development and stores the keys unencrypted in the specified data file.**

5. Set the principal key for the database using the `pg_tde_set_principal_key` function.

```sql
-- pg_tde_set_principal_key(principal_key_name, provider_name);
SELECT pg_tde_set_principal_key('my-principal-key','file');
-- pg_tde_set_principal_key_using_database_key_provider(principal_key_name, provider_name);
SELECT pg_tde_set_principal_key_using_database_key_provider('my-principal-key','file');
```

6. Specify `tde_heap_basic` access method during table creation
Expand Down
4 changes: 2 additions & 2 deletions contrib/pg_tde/documentation/docs/external-parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ To use the file provider with a file location specified by the `remote` method,
use the following command:

```
SELECT pg_tde_add_key_provider_file(
SELECT pg_tde_add_database_key_provider_file(
'file-provider',
json_object( 'type' VALUE 'remote', 'url' VALUE 'http://localhost:8888/hello' )
);"
Expand All @@ -24,7 +24,7 @@ SELECT pg_tde_add_key_provider_file(
Or to use the `file` method, use the following command:

```
SELECT pg_tde_add_key_provider_file(
SELECT pg_tde_add_database_key_provider_file(
'file-provider',
json_object( 'type' VALUE 'remote', 'path' VALUE '/tmp/datafile-location' )
);"
Expand Down
54 changes: 27 additions & 27 deletions contrib/pg_tde/documentation/docs/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ The following functions are also provided for easier management of functionality

Use these functions to grant or revoke permissions to manage permissions for the current database. They enable or disable all functions related to the providers and keys on the current database:

* `pg_tde_grant_local_key_management_to_role(role)`
* `pg_tde_revoke_local_key_management_from_role(role)`
* `pg_tde_grant_database_key_management_to_role(role)`
* `pg_tde_revoke_database_key_management_from_role(role)`

### Global scope key management

Expand Down Expand Up @@ -72,7 +72,7 @@ You can change an existing key provider using the provided functions, which are

There are two functions to change existing providers: one to change a provider in the current database, and another one to change a provider in the global scope.

* `pg_tde_change_key_provider_<type>('provider-name', <provider specific parameters>)`
* `pg_tde_change_database_key_provider_<type>('provider-name', <provider specific parameters>)`
* `pg_tde_change_global_key_provider_<type>('provider-name', <provider specific parameters>)`

When you change a provider, the referred name must exist in the database local or a global scope.
Expand All @@ -90,14 +90,14 @@ The Vault provider connects to a HashiCorp Vault or an OpenBao server, and store
Use the following functions to add the Vault provider:

```
SELECT pg_tde_add_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path');
SELECT pg_tde_add_database_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path');
SELECT pg_tde_add_global_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path');
```

These functions change the Vault provider:

```
SELECT pg_tde_change_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path');
SELECT pg_tde_change_database_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path');
SELECT pg_tde_change_global_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path');
```

Expand All @@ -121,14 +121,14 @@ The KMIP provider uses a remote KMIP server.
Use these functions to add a KMIP provider:

```
SELECT pg_tde_add_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
SELECT pg_tde_add_database_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
SELECT pg_tde_add_global_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
```

These functions change the KMIP provider:

```
SELECT pg_tde_change_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
SELECT pg_tde_change_database_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
SELECT pg_tde_change_global_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
```

Expand Down Expand Up @@ -156,14 +156,14 @@ This function is intended for development or quick testing, and stores the keys
Add a local keyfile provider:

```
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/key/provider/data.file');
SELECT pg_tde_add_database_key_provider_file('provider-name','/path/to/the/key/provider/data.file');
SELECT pg_tde_add_global_key_provider_file('provider-name','/path/to/the/key/provider/data.file');
```

Change a local keyfile provider:

```
SELECT pg_tde_change_key_provider_file('provider-name','/path/to/the/key/provider/data.file');
SELECT pg_tde_change_database_key_provider_file('provider-name','/path/to/the/key/provider/data.file');
SELECT pg_tde_change_global_key_provider_file('provider-name','/path/to/the/key/provider/data.file');
```

Expand All @@ -178,7 +178,7 @@ All parameters can be either strings, or JSON objects [referencing remote parame

These functions delete an existing provider in the current database or in the global scope:

* `pg_tde_delete_key_provider('provider-name)`
* `pg_tde_delete_database_key_provider('provider-name)`
* `pg_tde_delete_global_key_provider('provider-name)`

You can only delete key providers that are not currently in use. An error is returned if the current principal key is using the provider you are trying to delete.
Expand All @@ -189,7 +189,7 @@ If the use of global key providers is enabled via the `pg_tde.inherit_global` GU

These functions list the details of all key providers for the current database or for the global scope, including all configuration values:

* `pg_tde_list_all_key_providers()`
* `pg_tde_list_all_database_key_providers()`
* `pg_tde_list_all_global_key_providers()`

**All configuration values include possibly sensitive values, such as passwords. Never specify these directly, use the remote configuration option instead.**
Expand All @@ -201,12 +201,12 @@ Use these functions to create a new principal key for a specific scope such as a

Princial keys are stored on key providers by the name specified in this function - for example, when using the Vault provider, after creating a key named "foo", a key named "foo" will be visible on the Vault server at the specified mount point.

### pg_tde_set_principal_key
### pg_tde_set_principal_key_using_database_key_provider

Creates or rotates the principal key for the current database using the specified database key provider and key name.

```
SELECT pg_tde_set_principal_key('name-of-the-principal-key','provider-name','ensure_new_key');
SELECT pg_tde_set_principal_key_using_database_key_provider('name-of-the-principal-key','provider-name','ensure_new_key');
```

The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation:
Expand All @@ -215,12 +215,12 @@ SELECT pg_tde_set_principal_key('name-of-the-principal-key','provider-name','ens
If the provider already stores a key by that name, the function returns an error.
* If set to `false`, an existing principal key may be reused.

### pg_tde_set_global_principal_key
### pg_tde_set_principal_key_using_global_key_provider

Creates or rotates the principal key for the current database using the specified global key provider and key name.

```
SELECT pg_tde_set_global_principal_key('name-of-the-principal-key','provider-name','ensure_new_key');
SELECT pg_tde_set_principal_key_using_global_key_provider('name-of-the-principal-key','provider-name','ensure_new_key');
```

The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation:
Expand All @@ -229,12 +229,12 @@ SELECT pg_tde_set_global_principal_key('name-of-the-principal-key','provider-nam
If the provider already stores a key by that name, the function returns an error.
* If set to `false`, an existing principal key may be reused.

### pg_tde_set_server_principal_key
### pg_tde_set_server_principal_key_using_global_key_provider

Creates or rotates the global principal key using the specified key provider. Use this function to set a principal key for WAL encryption.
Creates or rotates the server principal key using the specified global key provider. Use this function to set a principal key for WAL encryption.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the server key? Maybe rewrite to what this key is used for? For WAL encryption, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now only for WAL encryption, but in future it may be used for something else. I guess for system tables encryption. In general its purpose to encrypt entities that has server scope availability.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thanks, I believe it's time to explain existing scopes and their usage. In a separate PR most likely

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely agree


```
SELECT pg_tde_set_server_principal_key('name-of-the-principal-key','provider-name','ensure_new_key');
SELECT pg_tde_set_server_principal_key_using_global_key_provider('name-of-the-principal-key','provider-name','ensure_new_key');
```

The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation:
Expand All @@ -244,14 +244,14 @@ The `ensure_new_key` parameter instructs the function how to handle a principal
* If set to `false`, an existing principal key may be reused.


### pg_tde_set_default_principal_key
### pg_tde_set_default_principal_key_using_global_key_provider

Creates or rotates the default principal key for the server using the specified key provider.
Creates or rotates the default principal key for the server using the specified global key provider.

The default key is automatically used as a principal key by any database that doesn't have a specific principal key already created when the first encrypted database object is created.

```
SELECT pg_tde_set_default_principal_key('name-of-the-principal-key','provider-name','ensure_new_key');
SELECT pg_tde_set_default_principal_key_using_global_key_provider('name-of-the-principal-key','provider-name','ensure_new_key');
```

The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation:
Expand Down Expand Up @@ -288,12 +288,12 @@ Displays information about the principal key for the current database, if it exi
SELECT pg_tde_principal_key_info()
```

### pg_tde_global_principal_key_info
### pg_tde_server_principal_key_info

Displays information about the principal key for the global scope, if exists.
Displays information about the principal key for the server scope, if exists.

```
SELECT pg_tde_global_principal_key_info()
SELECT pg_tde_server_principal_key_info()
```

### pg_tde_verify_principal_key
Expand All @@ -312,9 +312,9 @@ If any of the above checks fail, the function reports an error.
SELECT pg_tde_verify_principal_key()
```

### pg_tde_verify_global_principal_key
### pg_tde_verify_server_principal_key

This function checks that the global scope has a properly functional encryption setup, which means:
This function checks that the server scope has a properly functional encryption setup, which means:

* A key provider is configured
* The key provider is accessible using the specified configuration
Expand All @@ -325,5 +325,5 @@ This function checks that the global scope has a properly functional encryption
If any of the above checks fail, the function reports an error.

```
SELECT pg_tde_verify_principal_key()
SELECT pg_tde_verify_server_principal_key()
```
16 changes: 8 additions & 8 deletions contrib/pg_tde/documentation/docs/multi-tenant-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ You must do these steps for every database where you have created the extension.
For testing purposes, you can use the PyKMIP server which enables you to set up required certificates. To use a real KMIP server, make sure to obtain the valid certificates issued by the key management appliance.

```
SELECT pg_tde_add_key_provider_kmip('provider-name','kmip-addr', 5696, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
SELECT pg_tde_add_database_key_provider_kmip('provider-name','kmip-addr', 5696, '/path_to/server_certificate.pem', '/path_to/client_key.pem');
```

where:
Expand All @@ -75,15 +75,15 @@ You must do these steps for every database where you have created the extension.
<i warning>:material-information: Warning:</i> This example is for testing purposes only:

```
SELECT pg_tde_add_key_provider_kmip('kmip','127.0.0.1', 5696, '/tmp/server_certificate.pem', '/tmp/client_key_jane_doe.pem');
SELECT pg_tde_add_database_key_provider_kmip('kmip','127.0.0.1', 5696, '/tmp/server_certificate.pem', '/tmp/client_key_jane_doe.pem');
```

=== "With HashiCorp Vault"

The Vault server setup is out of scope of this document.

```sql
SELECT pg_tde_add_key_provider_vault_v2('provider-name','root_token','url','mount','ca_path');
SELECT pg_tde_add_database_key_provider_vault_v2('provider-name','root_token','url','mount','ca_path');
```

where:
Expand All @@ -96,28 +96,28 @@ You must do these steps for every database where you have created the extension.
<i warning>:material-information: Warning:</i> This example is for testing purposes only:

```
SELECT pg_tde_add_key_provider_file_vault_v2('my-vault','http://vault.vault.svc.cluster.local:8200,'secret/data','hvs.zPuyktykA...example...ewUEnIRVaKoBzs2', NULL);
SELECT pg_tde_add_database_key_provider_file_vault_v2('my-vault','http://vault.vault.svc.cluster.local:8200,'secret/data','hvs.zPuyktykA...example...ewUEnIRVaKoBzs2', NULL);
```

=== "With a keyring file"

This setup is intended for development and stores the keys unencrypted in the specified data file.

```sql
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/keyring/data.file');
SELECT pg_tde_add_database_key_provider_file('provider-name','/path/to/the/keyring/data.file');
```

<i warning>:material-information: Warning:</i> This example is for testing purposes only:

```sql
SELECT pg_tde_add_key_provider_file('file-keyring','/tmp/pg_tde_test_local_keyring.per');
SELECT pg_tde_add_database_key_provider_file('file-keyring','/tmp/pg_tde_test_local_keyring.per');
```


2. Add a principal key

```sql
SELECT pg_tde_set_principal_key('name-of-the-principal-key', 'provider-name','ensure_new_key');
SELECT pg_tde_set_principal_key_using_database_key_provider('name-of-the-principal-key', 'provider-name','ensure_new_key');
```

where:
Expand All @@ -129,7 +129,7 @@ You must do these steps for every database where you have created the extension.
<i warning>:material-information: Warning:</i> This example is for testing purposes only:

```sql
SELECT pg_tde_set_principal_key('test-db-master-key','file-vault','ensure_new_key');
SELECT pg_tde_set_principal_key_using_database_key_provider('test-db-master-key','file-vault','ensure_new_key');
```

The key is auto-generated.
Expand Down
4 changes: 2 additions & 2 deletions contrib/pg_tde/documentation/docs/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Load the `pg_tde` at startup time. The extension requires additional shared memo
2. Add a default principal key

```sql
SELECT pg_tde_set_default_principal_key('name-of-the-principal-key','provider-name','ensure_new_key');
SELECT pg_tde_set_default_principal_key_using_global_key_provider('name-of-the-principal-key','provider-name','ensure_new_key');
```

where:
Expand All @@ -123,7 +123,7 @@ Load the `pg_tde` at startup time. The extension requires additional shared memo
<i warning>:material-information: Warning:</i> This example is for testing purposes only. Replace the key name and provider name with your values:

```sql
SELECT pg_tde_set_global_principal_key('test-db-master-key','file-vault','ensure_new_key');
SELECT pg_tde_set_principal_key_using_global_key_provider('test-db-master-key','file-vault','ensure_new_key');
```

The key is auto-generated.
Expand Down
4 changes: 2 additions & 2 deletions contrib/pg_tde/documentation/docs/wal-encryption.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Here's what to do:
<i warning>:material-information: Warning:</i> This example is for testing purposes only:

```
SELECT pg_tde_add_key_global_provider_kmip('kmip','127.0.0.1', 5696, '/tmp/server_certificate.pem', '/tmp/client_key_jane_doe.pem');
SELECT pg_tde_add_key_using_global_key_provider_kmip('kmip','127.0.0.1', 5696, '/tmp/server_certificate.pem', '/tmp/client_key_jane_doe.pem');
```

=== "With HashiCorp Vault"
Expand Down Expand Up @@ -61,7 +61,7 @@ Here's what to do:
3. Create principal key

```sql
SELECT pg_tde_set_server_principal_key('principal-key', 'provider-name');
SELECT pg_tde_set_server_principal_key_using_global_key_provider('principal-key', 'provider-name');
```

4. Enable WAL level encryption using the `ALTER SYSTEM` command. You need the privileges of the superuser to run this command:
Expand Down
Loading