Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation updates for variables/tools and a few other things #108

Merged
merged 7 commits into from
Mar 27, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions contrib/pg_tde/documentation/docs/command-line-tools.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Command line tools

The `pg_tde` extension provides new command line tools and modifies some existing tools to work with encrypted WAL and tables.

## pg_tde_change_key_provider

A tool for modifying the configuration of a key provider, possibly also changing its type.

This tool edits the configuration files directly, ignoring permissions or running `postgres` processes.

Its only intended use is to fix servers that can't be started up because of inaccessible key providers.

For example, you restore from an old backup and the address of the key provider changed in the meantime. Use this tool can to correct the configuration, allowing the server to start up.

Use this tool **only when the server is offline.** To modify the key provider configuration when the server is up and running, use the [`pg_tde_change_key_provider_<type>`](fucntions.md#change-an-existing-provider) SQL functions.

### Usage

To modify the key provider configuration, specify all parameters depending on the provider type, in the same way as you do when using the [`pg_tde_change_key_provider_<type>`](fucntions.md#change-an-existing-provider) SQL functions.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To modify the key provider configuration, specify all parameters depending on the provider type, in the same way as you do when using the [`pg_tde_change_key_provider_<type>`](fucntions.md#change-an-existing-provider) SQL functions.
To modify the key provider configuration, specify all parameters depending on the provider type in the same way as you do when using the [`pg_tde_change_key_provider_<type>`](fucntions.md#change-an-existing-provider) SQL functions.

I think no need for a comma here


The general syntax is the following:

```
pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> <new_provider_type> <provider_parameters...>
```

where :

* [optional] `<datadir>` is the data directory. When not specified, `pg_de` uses the `$PGDATA` value.
* `<provider_name>` is the name you assigned to the key provider
* `<new_provider_type>` can be a `file`, `vault` or `kmip`.
* `<dbOid>`


Depending on the provider type, the additional parameters are:

```
pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> file <filename>
pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> vault <token> <url> <mount_path> [<ca_path>]
pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> kmip <host> <port> <cert_path> [<ca_path>]
```

## pg_waldump

[`pg_waldump`](https://www.postgresql.org/docs/current/pgwaldump.html) is a tool to display a human-readable rendering of the write-ahead log of a PostgreSQL database cluster.

To read encrypted WAL records, `pg_waldump` supports the following additional arguments:

* `keyring_path`: the directory where keyring config files for WAL are stored. These files are:

* `pg_tde.map`,
* `pg_tde.dat`,
* `pg_tde_keyrings`

`pg_waldump` will not try to decrypt WAL if not set.

## pg_checksums

`pg_checksums` is not able to calculate checksums for encrypted files.
It skips encrypted files, and reports this in the output.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
It skips encrypted files, and reports this in the output.
It skips encrypted files and reports this in the output.

No need for a comma here

10 changes: 5 additions & 5 deletions contrib/pg_tde/documentation/docs/decrypt.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Decrypt an encrypted table
# Remove the encryption from an encrypted table

## Method 1. Change the access method

If you encrypted a table with the `tde_heap` access method and need to decrypt it, run the following command against the desired table (`mytable` in the example below):
If you encrypted a table with the `tde_heap` access method and need to remove the encryption from it, run the following command against the desired table (`mytable` in the example below):

```
ALTER TABLE mytable SET ACCESS METHOD heap;
```

Note that the `SET ACCESS METHOD` command drops hint bits and this may affect the performance. Running a plain `SELECT count(*)` or `VACUUM` commands on the entire table will check every tuple for visibility and set its hint bits. Therefore, after executing the `ALTER TABLE` command, run a simple `count(*)` on your tables:
Note that the `SET ACCESS METHOD` command drops hint bits and this may affect performance. Running a plain `SELECT count(*)` or `VACUUM` commands on the entire table will check every tuple for visibility and set its hint bits. Therefore, after executing the `ALTER TABLE` command, run a simple `count(*)` on your tables:

```
SELECT count(*) FROM mytable;
Expand All @@ -24,9 +24,9 @@ The output returns `f` meaning that the table is no longer encrypted.



## Method 2. Create a new unencrypted table on the base of the encrypted one
## Method 2. Create a new not encrypted table on the base of the encrypted one

Alternatively, you can create a new unencrypted table with the same structure and data as the initial table. For example, the original encrypted table is `EncryptedCustomers`. Use the following command to create a new table `Customers`:
Alternatively, you can create a new not encrypted table with the same structure and data as the initial table. For example, the original encrypted table is `EncryptedCustomers`. Use the following command to create a new table `Customers`:

```
CREATE TABLE Customers AS
Expand Down
30 changes: 21 additions & 9 deletions contrib/pg_tde/documentation/docs/uninstall.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,39 @@
# Uninstall `pg_tde`

If you no longer wish to use TDE in your deployment, you can remove the `pg_tde` extension. To do that, your user must have the superuser privileges or a database owner.
If you no longer wish to use TDE in your deployment, you can remove the `pg_tde` extension. To do that, your user must have the superuser privileges, or a database owner privileges in case you only want to remove it from a single database.

Here's how to do it:

1. Drop the extension using the `DROP EXTENSION` with `CASCADE` command.
1. Drop the extension using the `DROP EXTENSION` command:

<i warning>:material-alert: Warning:</i> The use of the CASCADE parameter deletes all tables that were created in the database with `pg_tde` enabled and also all dependencies upon the encrypted table (e.g. foreign keys in a non-encrypted table used in the encrypted one).
```
DROP EXTENSION pg_tde;
```

```
DROP EXTENSION pg_tde CASCADE
```
This command will fail if there are still encrypted tables in the database.

2. Run the `DROP EXTENSION` command against every database where you have enabled the `pg_tde` extension
In this case, the dependent objects have to be dropped manually. Alternatively, you can run the `DROP EXTENSION ... CASCADE` command to drop all dependent objects automatically.

3. Modify the `shared_preload_libraries` and remove the 'pg_tde' from it. Use the `ALTER SYSTEM` command for this purpose
Note that the `DROP EXTENSION` command does not delete the `pg_tde` data files related to the database.

2. Run the `DROP EXTENSION` command against every database where you have enabled the `pg_tde` extension, if the goal is to completely remove the extension. This also includes the template databases, in case `pg_tde` was previously enabled there.

3. Remove any reference to `pg_tde` GUC variables from the PostgreSQL configuration file.

4. Modify the `shared_preload_libraries` and remove the 'pg_tde' from it. Use the `ALTER SYSTEM` command for this purpose, or edit the configuration file.

!!! warning

Once `pg_tde` is removed from the `shared_preload_libraries`, reading any leftover encrypted files will fail. Removing the extension from the `shared_preload_libraries` is also possible if the extension is still installed in some databases.

Make sure to only do this if the server has no encrypted files in its data directory.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Make sure to only do this if the server has no encrypted files in its data directory.
Make sure to do this only if the server has no encrypted files in its data directory.


4. Start or restart the `postgresql` cluster to apply the changes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this now be item 5. instead of 4. twice?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


* On Debian and Ubuntu:

```sh
sudo systemctl restart postgresql-17
sudo systemctl restart postgresql
```

* On RHEL and derivatives
Expand Down
58 changes: 58 additions & 0 deletions contrib/pg_tde/documentation/docs/variables.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# GUC Variables

The `pg_tde` extension provides GUC variables to configure the behaviour of the extension:

## pg_tde.wal_encrypt

**Type** - boolean <br>
**Default** - off

A `boolean` variable controlling if WAL writes are encrypted or not.

Changing this variable requires a server restart, and can only be set at the server level.

WAL encryption is controlled globally. If enabled, all WAL writes are encrypted in the entire PostgreSQL cluster.

This variable only controls new writes to the WAL, it doesn't affect existing WAL records.

`pg_tde` is always capable of reading existing encrypted WAL records, as long as the keys used for the encryption are still available.

Enabling WAL encryption requires a configured global principal key. Refer to the [WAL encryption configuration](wal-encription.md) documentation for more information.

## pg_tde.enforce_encryption

**Type** - boolean <br>
**Default** - off

A `boolean` variable controlling if the creation of new not encrypted tables is enabled or not.

If enabled, `CREATE TABLE` statements will fail unless they use the `tde_heap` access method.

Similarly, `ALTER TABLE <x> SET ACCESS METHOD` is only allowed, if the access method is `tde_heap`.

Other DDL operations are still allowed. For example other `ALTER` commands are allowed on unencrypted tables, as long as the access method isn't changed.

You can set this variable at the following levels.

* global - for the entire PostgreSQL cluster
* database - for the current database
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for specific databases

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to know what databases is it set for?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have to know, it's managed by postgres: https://www.postgresql.org/docs/current/sql-alterdatabase.html

It's the ALTER DATABASE ... SET variable = value; syntax.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also why I didn't write a too detailed description about it. This is a common postgresql concept for most GUC variables, we shouldn't have to explain it to anybody familiar with postgres.

* user -
* session - for the current user session.

Setting or changing the value requires superuser permissions.

## pg_tde.inherit_global_providers

**Type** - boolean <br>
**Default** - on

A `boolean` variable controlling if databases can use global key providers for storing principal keys.

This can be set at global, database, user or session level, but changing the value requires superuser permissions.

If disabled, functions that change the key providers can only work with database local key providers.
In this case, the default principal key, if set, is also disabled.

This doesn't affect existing uses of global keys. It only prevents the creation of new principal keys using global providers.

The default value is true.
2 changes: 2 additions & 0 deletions contrib/pg_tde/documentation/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,8 @@ nav:
- "3. Configure WAL encryption (tech preview)": wal-encryption.md
- "4. Test TDE": "test.md"
- functions.md
- command-line-tools.md
- variables.md
- Concepts:
- "What is TDE": tde.md
- table-access-method.md
Expand Down