Tells if a table is using the pg_tde access method or not.
Tells if a table is encrypted using the tde_heap access method or not.
To verify a table encryption, run the following statement:
SELECT pg_tde_is_encrypted('table_name');
You can also verify if the table in a custom schema is encrypted. Pass teh schema name for the function as follows:
+SELECT pg_tde_is_encrypted('schema.table_name');
+pg_tde documentation","text":"pg_tde is the open source PostgreSQL extension that provides Transparent Data Encryption (TDE) to protect data at rest. This ensures that the data stored on disk is encrypted, and no one can read it without the proper encryption keys, even if they gain access to the physical storage media.
You can configure encryption differently for each database, encrypting specific tables in some databases with different encryption keys while keeping others unencrypted.
Lear more what is Transparent Data Encryption and why you need it.
Important
This extension is in the experimental phase and is under active development. It is not meant for production use yet.
What\u2019s new
"},{"location":"index.html#whats-encrypted","title":"What\u2019s encrypted:","text":"pg_tde encrypts the following:
Warning: Note that introducing encryption/decryption affects performance. Our benchmark tests show less than 10% performance overhead for most situations. However, in some specific applications such as those using JSONB operations, performance degradation might be higher.
"},{"location":"index.html#versions-and-supported-postgresql-deployments","title":"Versions and supported PostgreSQL deployments","text":"The pg_tde extension comes in two distinct versions with specific access methods to encrypt the data. These versions are database-specific and differ in terms of what they encrypt and with what access method. Each version is characterized by the database it supports, the access method it provides, and the scope of encryption it offers.
Version for Percona Server for PostgreSQL
This pg_tde version is based on and supported for Percona Server for PostgreSQL 17.x - an open source binary drop-in replacement for PostgreSQL Community. It provides the tde_heap access method and offers full encryption capabilities.
Community version
This version is supported for PostgreSQL Community 16 and 17, and Percona Distribution for PostgreSQL 16. It provides the tde_heap_basic access method, offering limited encryption features. The limitations are in encrypting WAL data only for tables created using the extension and no support of index encryption nor logical replication.
The answer is pretty straightforward: for data sets where indexing is not mandatory or index encryption is not required, use the community version and the tde_heap_basic access method. Check the upstream documentation how to get started.
Otherwise, enjoy full encryption with the Percona Server for PostgreSQL version and the tde_heap access method.
Still not sure? Contact our experts to find the best solution for you.
Get started
"},{"location":"index.html#future-releases","title":"Future releases","text":"The following is planned for future releases of pg_tde:
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"apt.html","title":"Installpg_tde on Debian or Ubuntu","text":"The packages for the tech preview pg_tde are available in the experimental repository for Percona Distribution for PostgreSQL 17.
Check the list of supported platforms.
This tutorial shows how to install pg_tde with Percona Distribution for PostgreSQL.
You need the percona-release repository management tool that enables the desired Percona repository for you.
You need the following dependencies to install percona-release:
wgetgnupg2curllsb-releaseInstall them with the following command:
sudo apt-get install -y wget gnupg2 curl lsb-release\nFetch the percona-release package
sudo wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb\nInstall percona-release
sudo dpkg -i percona-release_latest.generic_all.deb\nEnable the Percona Distribution for PostgreSQL repository
Percona provides two repositories for Percona Distribution for PostgreSQL. We recommend enabling the Major release repository to timely receive the latest updates. Since the tde_heap access method is still in the experimental stage, the pg_tde package is currently available from the experimental repository.
$ sudo percona-release enable ppg-17.0 experimental\nUpdate the local cache
sudo apt-get update\npg_tde","text":"After all preconditions are met, install the extension.
Install Percona Distribution for PostgreSQL.
Run the following command to install Percona Distribution for PostgreSQL and the required packages:
sudo apt-get install -y percona-postgresql-17 percona-postgresql-contrib percona-postgresql-server-dev-all\nInstall pg_tde packages
sudo apt-get install percona-postgresql-17-pg-tde\nSetup
"},{"location":"apt.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"contribute.html","title":"Contributing guide","text":"Welcome to pg_tde - the Transparent Data Encryption extension for PostgreSQL!
We\u2019re glad that you would like to become a community member and contribute to this project.
You can contribute in one of the following ways:
Before submitting code contributions, we ask you to complete the following prerequisites.
"},{"location":"contribute.html#1-sign-the-cla","title":"1. Sign the CLA","text":"Before you can contribute, we kindly ask you to sign our Contributor License Agreement (CLA). You can do this in on click using your GitHub account.
Note: You can sign it later, when submitting your first pull request. The CLA assistant validates the PR and asks you to sign the CLA to proceed.
"},{"location":"contribute.html#2-code-of-conduct","title":"2. Code of Conduct","text":"Please make sure to read and agree to our Code of Conduct.
"},{"location":"contribute.html#submitting-a-pull-request","title":"Submitting a pull request","text":"All bug reports, enhancements and feature requests are tracked in GitHub issues. Though not mandatory, we encourage you to first check for a bug report among the issues and in the PR list: perhaps the bug has already been addressed.
For feature requests and enhancements, we do ask you to create a GitHub issue, describe your idea and discuss the design with us. This way we align your ideas with our vision for the product development.
If the bug hasn\u2019t been reported / addressed, or we\u2019ve agreed on the enhancement implementation with you, do the following:
Make your changes. Please follow the following guidelines to improve code readability:
Build pg_tde and test your changes locally.
When you work, you should periodically run tests to check that your changes don\u2019t break existing code.
To run the tests, use the following command:
cd pg_tde\nmake USE_PGXS=1 installcheck\nYou can run tests on your local machine with whatever operating system you have. After you submit the pull request, we will check your patch on multiple operating systems.
"},{"location":"contribute.html#contribute-to-documentation","title":"Contribute to documentation","text":"pg_tde documentation is written in Markdown language, so you can edit it online via GitHub. If you wish to have more control over the doc process, jump to how to edit documentation locally.
Before you start, learn what git, MkDocs and Docker are and what Markdown is and how to write it. For your convenience, there\u2019s also a cheat sheet to help you with the syntax.
The doc files are in the docs directory.
.md file of the page opens in GitHub editor in your browser. If you haven\u2019t worked with the repository before, GitHub creates a fork of it for you.This option is for users who prefer to work from their computer and / or have the full control over the documentation process.
The steps are the following:
git clone git@github.com:<your-name>/pg_tde.git\n\n3. Change the directory to ``pg_tde`` and add the remote upstream repository:\n\n```sh\ngit remote add upstream git@github.com:percona/pg_tde.git\ngit fetch upstream\ngit merge upstream/main\ngit checkout -b <my_branch>\nCommit your changes. The commit message guidelines will help you with writing great commit messages
Open a pull request to Percona
To verify how your changes look, generate the static site with the documentation. This process is called building. You can do it in these ways: - Use Docker - Install MkDocs and build locally
"},{"location":"contribute.html#use-docker","title":"Use Docker","text":"cd documentation\ndocker run --rm -v $(pwd):/docs perconalab/pmm-doc-md mkdocs build\nsite directory and open the index.html file to see the documentation.If you want to see the changes as you edit the docs, use this command instead:
cd documentation\ndocker run --rm -v $(pwd):/docs -p 8000:8000 perconalab/pmm-doc-md mkdocs serve --dev-addr=0.0.0.0:8000\nWait until you see INFO - Start detecting changes, then enter 0.0.0.0:8000 in the browser\u2019s address bar. The documentation automatically reloads after you save the changes in source files.
Install Python.
Install MkDocs and required extensions:
pip install -r requirements.txt\nBuild the site:
cd documentation\nmkdocs build\nOpen site/index.html
Or, to run the built-in web server:
cd documentation\nmkdocs serve\nView the site at http://0.0.0.0:8000
"},{"location":"contribute.html#build-pdf-file","title":"Build PDF file","text":"To build a PDF version of the documentation, do the following:
Disable displaying the last modification of the page:
export ENABLED_GIT_REVISION_DATE=false\nBuild the PDF file:
ENABLE_PDF_EXPORT=1 mkdocs build -f mkdocs-pdf.yml\nThe PDF document is in the site/pdf folder.
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"decrypt.html","title":"Decrypt an encrypted table","text":""},{"location":"decrypt.html#method-1-change-the-access-method","title":"Method 1. Change the access method","text":"If you encrypted a table with the tde_heap or tde_heap_basic access method and need to decrypt it, run the following command against the desired table (mytable in the example below):
ALTER TABLE mytable SET access method heap;\nCheck that the table is not encrypted:
SELECT pg_tde_is_encrypted('mytable');\nThe output returns f meaning that the table is no longer encrypted.
In the same way you can re-encrypt the data with the tde_heap_basic access method.
ALTER TABLE mytable SET access method tde_heap_basic;\nNote that the indexes and WAL files will no longer be encrypted.
"},{"location":"decrypt.html#method-2-create-a-new-unencrypted-table-on-the-base-of-the-encrypted-one","title":"Method 2. Create a new unencrypted table on the base of the encrypted one","text":"Alternatively, you can create a new unencrypted table with the same structure and data as the initial table. For example, the original encrypted table is EncryptedCustomers. Use the following command to create a new table Customers:
CREATE TABLE Customers AS\nSELECT * FROM EncryptedCustomers;\nThe new table Customers inherits the structure and the data from EncryptedCustomers.
(Optional) If you no longer need the EncryptedCustomers table, you can delete it.
DROP TABLE EncryptedCustomers;\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"external-parameters.html","title":"Use external reference to parameters","text":"To allow storing secrets or any other parameters in a more secure, external location, pg_tde allows users to specify an external reference instead of hardcoded parameters.
In Alpha1 version, pg_tde supports the following external storage methods:
file, which just stores the data in a simple file specified by a path. The file should be readable to the postgres process.remote, which uses a HTTP request to retrieve the parameter from the specified url.To use the file provider with a file location specified by the remote method, use the following command:
SELECT pg_tde_add_key_provider_file(\n 'file-provider', \n json_object( 'type' VALUE 'remote', 'url' VALUE 'http://localhost:8888/hello' )\n );\"\nOr to use the file method, use the following command:
SELECT pg_tde_add_key_provider_file(\n 'file-provider', \n json_object( 'type' VALUE 'remote', 'path' VALUE '/tmp/datafile-location' )\n );\"\nAny parameter specified to the add_key_provider function can be a json_object instead of the string, similar to the above examples.
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"faq.html","title":"FAQ","text":""},{"location":"faq.html#why-do-i-need-tde","title":"Why do I need TDE?","text":"Encrypting a hard drive encrypts all data including system and application files that are there. However, disk encryption doesn\u2019t protect your data after the boot-up of your system. During runtime, the files are decrypted with disk-encryption.
TDE focuses specifically on data files and offers a more granular control over encrypted data. It also ensures that files are encrypted on disk during runtime and when moved to another system or storage.
Consider using TDE and storage-level encryption together to add another layer of data security
"},{"location":"faq.html#is-tde-enough-to-ensure-data-security","title":"Is TDE enough to ensure data security?","text":"No. TDE is an additional layer to ensure data security. It protects data at rest. Consider introducing also these measures:
If you lose encryption keys, especially, the principal key, the data is lost. That\u2019s why it\u2019s critical to back up your encryption keys securely.
"},{"location":"faq.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"functions.html","title":"Functions","text":"The pg_tde extension provides the following functions:
Creates a new key provider for the database using a local file.
This function is intended for development, and stores the keys unencrypted in the specified data file.
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/keyring/data.file');\nAll parameters can be either strings, or JSON objects referencing remote parameters.
"},{"location":"functions.html#pg_tde_add_key_provider_vault_v2","title":"pg_tde_add_key_provider_vault_v2","text":"Creates a new key provider for the database using a remote HashiCorp Vault server.
The specified access parameters require permission to read and write keys at the location.
SELECT pg_tde_add_key_provider_vault_v2('provider-name',:'secret_token','url','mount','ca_path');\nwhere:
url is the URL of the Vault servermount is the mount point where the keyring should store the keyssecret_token is an access token with read and write access to the above mount pointca_path is the path of the CA file used for SSL verificationAll parameters can be either strings, or JSON objects referencing remote parameters.
"},{"location":"functions.html#pg_tde_set_principal_key","title":"pg_tde_set_principal_key","text":"Sets the principal key for the database using the specified key provider.
The principal key name is also used for constructing the name in the provider, for example on the remote Vault server.
You can use this function only to a principal key. For changes in the principal key, use the pg_tde_rotate_principal_key function.
SELECT pg_tde_set_principal_key('name-of-the-principal-key', 'provider-name');\nCreates a new version of the specified principal key and updates the database so that it uses the new principal key version.
When used without any parameters, the function will just create a new version of the current database principal key, using the same provider:
SELECT pg_tde_rotate_principal_key();\nAlternatively, you can pass two parameters to the function, specifying both a new key name and a new provider name:
SELECT pg_tde_rotate_principal_key('name-of-the-new-principal-key', 'name-of-the-new-provider');\nBoth parameters support the NULL value, which means that the parameter won\u2019t be changed:
-- creates new principal key on the same provider as before\nSELECT pg_tde_rotate_principal_key('name-of-the-new-principal-key', NULL);\n\n-- copies the current principal key to a new provider\nSELECT pg_tde_rotate_principal_key(NULL, 'name-of-the-new-provider');\nTells if a table is using the pg_tde access method or not.
SELECT pg_tde_is_encrypted('table_name');\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"install.html","title":"Installation","text":""},{"location":"install.html#considerations","title":"Considerations","text":"You can use the following options to manage encryption keys:
Use the HashiCorp Vault server. This is the recommended approach. The Vault server configuration is out of scope of this document. We assume that you have the Vault server up and running. For the pg_tde configuration, you need the following information:
Use the local keyfile. This approach is rather used for development and testing purposes since the keys are stored unencrypted in the specified keyfile.
Install pg_tde using one of available installation methods:
The packages are available for the following operating systems:
Install on Debian or Ubuntu Install on RHEL or derivatives
To build pg_tde from source code, do the following
On Ubuntu/Debian: Install the following dependencies required for the build:
sudo apt install make gcc postgresql-server-dev-17 libcurl4-openssl-dev\nInstall Percona Distribution for PostgreSQL 17 or upstream PostgreSQL 17
If PostgreSQL is installed in a non standard directory, set the PG_CONFIG environment variable to point to the pg_config executable.
Clone the repository:
git clone git://github.com/percona/pg_tde\nCompile and install the extension
cd pg_tde\nmake USE_PGXS=1\nsudo make USE_PGXS=1 install\nNote
The steps below are for the pg_tde community version.
You can find Docker images built from the current main branch on Docker Hub. Images are built on top of postgres:17 official image.
To run pg_tde in Docker, use the following command:
docker run --name pg-tde -e POSTGRES_PASSWORD=mysecretpassword -d perconalab/pg_tde\nIt builds and adds pg_tde extension to PostgreSQL 17. The postgresql.conf contains the required modifications. The pg_tde extension is added to template1 so that all new databases automatically have the pg_tde extension loaded.
Keys are not created automatically. You must configure a key provider and a principal key for each database where you wish to use encrypted tables. See the instructions in the Setup section, starting with the 4th point, as the first 3 steps are already completed in the Docker image.
See Docker Docs on usage.
You can also build a Docker image manually with:
docker build . -f ./docker/Dockerfile -t your-image-name\nSetup
"},{"location":"install.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"setup.html","title":"Set uppg_tde","text":""},{"location":"setup.html#enable-extension","title":"Enable extension","text":"Load the pg_tde at the start time. The extension requires additional shared memory; therefore, add the pg_tde value for the shared_preload_libraries parameter and restart the postgresql instance.
Use the ALTER SYSTEM command from psql terminal to modify the shared_preload_libraries parameter.
ALTER SYSTEM SET shared_preload_libraries = 'pg_tde';\nStart or restart the postgresql instance to apply the changes.
$ sudo systemctl restart postgresql.service\n$ sudo systemctl restart postgresql-17\nCreate the extension using the CREATE EXTENSION command. You must have the privileges of a superuser or a database owner to use this command. Connect to psql as a superuser for a database and run the following command:
CREATE EXTENSION pg_tde;\nBy default, the pg_tde extension is created for the currently used database. To enable data encryption in other databases, you must explicitly run the CREATE EXTENSION command against them.
Tip
You can have the pg_tde extension automatically enabled for every newly created database. Modify the template template1 database as follows:
psql -d template1 -c 'CREATE EXTENSION pg_tde;'\nSet up a key provider for the database where you have enabled the extension.
With HashiCorp VaultWith keyring fileThe Vault server setup is out of scope of this document.
SELECT pg_tde_add_key_provider_vault_v2('provider-name',:'secret_token','url','mount','ca_path');\nwhere:
url is the URL of the Vault servermount is the mount point where the keyring should store the keyssecret_token is an access token with read and write access to the above mount pointca_path is the path of the CA file used for SSL verificationThis setup is intended for development and stores the keys unencrypted in the specified data file.
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/keyring/data.file');\nWarning: This example is for testing purposes only:
SELECT pg_tde_add_key_provider_file('file-vault','/tmp/pg_tde_test_local_keyring.per');\nAdd a principal key
SELECT pg_tde_set_principal_key('name-of-the-principal-key', 'provider-name');\nWarning: This example is for testing purposes only:
SELECT pg_tde_set_principal_key('test-db-master-key','file-vault');\nThe key is auto-generated.
Info: The key provider configuration is stored in the database catalog in an unencrypted table. See how to use external reference to parameters to add an extra security layer to your setup.
"},{"location":"setup.html#wal-encryption-configuration-tech-preview","title":"WAL encryption configuration (tech preview)","text":"After you enabled pg_tde and started the Percona Server for PostgreSQL, a principal key and internal keys for WAL encryption are created. They are stored in the data directory so that after WAL encryption is enabled, any process that requires access to WAL (a recovery or a checkpointer) can use them for decryption.
Now you need to instruct pg_tde to encrypt WAL files by configuring WAL encryption. Here\u2019s how to do it:
Use the ALTER SYSTEM SET command. You need the privileges of the superuser to run this command:
ALTER SYSTEM set pg_tde.wal_encrypt = on;\nRestart the server to apply the changes.
$ sudo systemctl restart postgresql.service\n$ sudo systemctl restart postgresql-17\nOn the server start
"},{"location":"setup.html#rotate-the-principal-key","title":"Rotate the principal key","text":"We highly recommend you to create your own keyring and rotate the principal key. This is because the default principal key is created from the local keyfile and is stored unencrypted.
Rotating the principal key means re-encrypting internal keys used for WAL encryption with the new principal key. This process doesn\u2019t stop the database operation meaning that reads and writes can take place as usual during key rotation.
Set up the key provider for WAL encryption
With HashiCorp VaultWith keyring fileSELECT pg_tde_add_key_provider_vault_v2('PG_TDE_GLOBAL','provider-name',:'secret_token','url','mount','ca_path');\nwhere:
PG_TDE_GLOBAL is the constant that defines the WAL encryption key provider-name is the name you define for the key providerurl is the URL of the Vault servermount is the mount point where the keyring should store the keyssecret_token is an access token with read and write access to the above mount pointca_path is the path of the CA file used for SSL verificationThis setup is intended for development and stores the keys unencrypted in the specified data file.
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/keyring/data.file');\nRotate the principal key. Don\u2019t forget to specify the PG_TDE_GLOBAL constant to rotate only the principal key for WAL.
SELECT pg_tde_rotate_principal_key('PG_TDE_GLOBAL', 'new-principal-key', 'provider-name');\nNow all WAL files are encrypted for both encrypted and unencrypted tables.
Verify the encryption by checking the pg_tde.wal_encrypt GUC (Grand Unified Configuration) parameter as follows:
SELECT name, setting FROM pg_settings WHERE name = 'pg_tde.wal_encrypt';\n name | setting\n--------------------+---------\n pg_tde.wal_encrypt | on\nTest TDE
"},{"location":"setup.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"switch.html","title":"Switch from Percona Server for PostgreSQL to PostgreSQL Community","text":"Percona Server for PostgreSQL and PostgreSQL Community are binary compatible and enable you to switch from one to another. Here\u2019s how:
tde_heap (tech preview feature) access method for encryption, either re-encrypt the data using the tde_heap_basic access method, or decrypt it completely If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"table-access-method.html","title":"Table access method","text":"A table access method is the way how PostgreSQL stores the data in a table. The default table access method is heap. PostgreSQL organizes data in a heap structure, meaning there is no particular order to the rows in the table. Each row is stored independently and identified by its unique row identifier (TID).
heap access method works","text":"Insertion: When a new row is inserted, PostgreSQL finds a free space in the tablespace and stores the row there.
Deletion: When a row is deleted, PostgreSQL marks the space occupied by the row as free, but the data remains until it is overwritten by a new insertion.
Updates: PostgreSQL handles updates by deleting the old row and inserting a new row with the updated values.
"},{"location":"table-access-method.html#custom-access-method","title":"Custom access method","text":"Custom access methods allow you to implement and define your own way of organizing data in PostgreSQL. This is useful if the default table access method doesn\u2019t meet your needs.
Custom access methods are typically available with PostgreSQL extensions. When you install an extension and enable it in PostgreSQL, a custom access method is created.
An example of such an approach is the tde_heap access method. It is automatically created only for the databases where you enabled the pg_tde extension and configured the key provider, enabling you to encrypt the data.
To use a custom access method, specify the USING clause for the CREATE TABLE command:
CREATE TABLE table_name (\n column1 data_type,\n column2 data_type,\n ...\n) USING tde_heap;\ntde_heap works","text":"The tde_heap access method works on top of the default heap access method and is a marker to point which tables require encryption. It uses the custom storage manager TDE SMGR, which becomes active only after you installed the pg_tde extension.
Every data modification operation is first sent to the Buffer Manager, which updates the buffer cache. Then, it is passed to the storage manager, which then writes it to disk. When a table requires encryption, the data is sent to the TDE storage manager, where it is encrypted before written to disk.
Similarly, when a client queries the database, the PostgreSQL core sends the request to the Buffer Manager which checks if the requested data is already in the buffer cache. If it\u2019s not there, the Buffer Manager requests the data from the storage manager. The TDE storage manager reads the encrypted data from disk, decrypts it and loads it to the buffer cache. The Buffer Manager sends the requested data to the PostgreSQL core and then to the client.
Thus, the encryption is done at the storage manager level.
"},{"location":"table-access-method.html#changing-the-default-table-access-method","title":"Changing the default table access method","text":"You can change the default table access method so that every table in the entire database cluster is created using the custom access method. For example, you can enable data encryption by default by defining the tde_heap as the default table access method.
However, consider the following before making this change:
CREATE statement will default to the specified table access method. pg_tde extension and configure the key provider for all databases before you modify the configuration. Otherwise PostgreSQL won\u2019t find the specified access method and will throw an error.Here\u2019s how you can set the new default table access method:
Add the access method to the default_table_access_method parameter.
Use the ALTER SYSTEM SET command. This requires superuser or ALTER SYSTEM privileges.
This example shows how to set the tde_heap access method. Replace it with the tde_heap_basic if needed.
ALTER SYSTEM SET default_table_access_method=tde_heap;\nEdit the postgresql.conf configuration file and add the value for the default_table_access_method parameter.
This example shows how to set the tde_heap access method. Replace it with the tde_heap_basic if needed.
default_table_access_method = 'tde_heap'\nYou can use the SET command to change the default table access method temporarily, for the current session.
Unlike modifying the postgresql.conf file or using the ALTER SYSTEM SET command, the changes you make via the SET command don\u2019t persist after the session ends.
You also don\u2019t need to have the superuser privileges to run the SET command.
You can run the SET command anytime during the session. This example shows how to set the tde_heap access method. Replace it with the tde_heap_basic if needed.
SET default_table_access_method = tde_heap;\nReload the configuration to apply the changes:
SELECT pg_reload_conf();\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"tde.html","title":"What is Transparent Data Encryption (TDE)","text":"Transparent Data Encryption is a technology to protect data at rest. The encryption process happens transparently in the background, without affecting database operations. Data is automatically encrypted as it\u2019s written to the disk and decrypted as it\u2019s read, all in real-time. Users and applications interact with the data as usual without noticing any difference.
"},{"location":"tde.html#how-does-it-work","title":"How does it work?","text":"To encrypt the data, two types of keys are used:
pg_tde is integrated with HashiCorp Vault server to store and manage principal keys. Only the back end KV Secrets Engine - Version 2 (API) is supported.
The encryption process is the following:
When a user creates an encrypted table using pg_tde, a new random key is generated for that table using the AES128 (AES-ECB) cipher algorithm. This key is used to encrypt all data the user inserts in that table. Eventually the encrypted data gets stored in the underlying storage.
The table itself is encrypted using the principal key. The principal key is stored externally in the Vault key management store.
Similarly when the user queries the encrypted table, the principal key is retrieved from the key store to decrypt the table. Then the same unique internal key for that table is used to decrypt the data, and unencrypted data gets returned to the user. So, effectively, every TDE table has a unique key, and each table key is encrypted using the principal key.
"},{"location":"tde.html#why-do-you-need-tde","title":"Why do you need TDE?","text":"Using TDE has the following benefits:
For organizations:
For DBAs:
See also
Percona Blog: Transparent Data Encryption (TDE)
"},{"location":"tde.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"test.html","title":"Test Transparent Data Encryption","text":"Enabling pg_tde extension for a database creates the table access method tde_heap . This access method enables you to encrypt the data.
Here\u2019s how to do it:
Create a table in the database for which you have enabled pg_tde using the tde_heap access method as follows:
CREATE TABLE <table_name> (<field> <datatype>) USING tde_heap;\nWarning: Example for testing purposes only:
CREATE TABLE albums (\nalbum_id INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY,\nartist_id INTEGER,\ntitle TEXT NOT NULL,\nreleased DATE NOT NULL\n) USING tde_heap;\nLearn more about table access methods and how you can enable data encryption by default in the Table access methods section.
To check if the data is encrypted, run the following function:
SELECT pg_tde_is_encrypted('table_name');\nThe function returns t if the table is encrypted and f - if not.
Rotate the principal key when needed:
SELECT pg_tde_rotate_principal_key(); -- uses automatic key versionin\n-- or\nSELECT pg_tde_rotate_principal_key('new-principal-key', NULL); -- specify new key name\n-- or\nSELECT pg_tde_rotate_principal_key('new-principal-key', 'new-provider'); -- changeprovider\nYou can encrypt an existing table. It requires rewriting the table, so for large tables, it might take a considerable amount of time.
ALTER TABLE table_name SET access method tde_heap;\nHint
If you no longer wish to use pg_tde or wish to switch to using the tde_heap_basic access method, see how you can decrypt your data.
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"uninstall.html","title":"Uninstallpg_tde","text":"If you no longer wish to use TDE in your deployment, you can remove the pg_tde extension. To do that, your user must have the privileges of the superuser or a database owner.
Here\u2019s how to do it:
DROP EXTENSION with CASCADE command. Warning: The use of the CASCADE parameter deletes all tables that were created in the database with pg_tde enabled and also all dependencies upon the encrypted table (e.g. foreign keys in a non-encrypted table used in the encrypted one).
DROP EXTENSION pg_tde CASCADE\nRun the DROP EXTENSION command against every database where you have enabled the pg_tde extension
Modify the shared_preload_libraries and remove the \u2018pg_tde\u2019 from it. Use the ALTER SYSTEM SET command for this purpose
Start or restart the postgre instance to apply the changes.
sudo systemctl restart postgre.service\nsudo systemctl restart postgre-17\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"yum.html","title":"Installpg_tde on Red Hat Enterprise Linux and derivatives","text":"The packages for the tech preview pg_tde are available in the experimental repository for Percona Distribution for PostgreSQL 17.
Check the list of supported platforms.
This tutorial shows how to install pg_tde with Percona Distribution for PostgreSQL.
For RHEL8/Oracle Linux 8/Rocky Linux 8, disable the postgresql and llvm-toolsetmodules:
sudo dnf module disable postgresql llvm-toolset\npercona-release","text":"You need the percona-release repository management tool that enables the desired Percona repository for you.
Install percona-release:
sudo yum -y install https://repo.percona.com/yum/percona-release-latest.noarch.rpm \nEnable the repository
sudo percona-release enable-only ppg-17.0 experimental\nYou need to install the percona-postgresql{{ pgversion }}-devel package when working with pg_tde. This package requires dependencies that are not part of the Distribution, but can be installed from the specific repositories:
```\n$ sudo dnf config-manager --set-enabled ol9_codeready_builder\n```\npercona-postgresql{{ pgversion }}-devel package","text":""},{"location":"yum.html#install-pg_tde","title":"Install pg_tde","text":"Install Percona Distribution for PostgreSQL 17 and the required packages, run the following command:
sudo yum -y install percona-postgresql-client-common percona-postgresql-common percona-postgresql-server-dev-all percona-postgresql17 percona-postgresql17-contrib percona-postgresql17-devel percona-postgresql17-libs\nInstall pg_tde packages
sudo yum install percona-pg_tde_17\nSetup
"},{"location":"yum.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"release-notes/release-notes.html","title":"pg_tde release notes","text":"pg_tde extension brings in Transparent Data Encryption (TDE) to PostgreSQL and enables you to keep sensitive data safe and secure.
Get started
"},{"location":"release-notes/release-notes.html#beta-2024-06-30","title":"Beta (2024-06-30)","text":"With this version, the access method for pg_tde extension is renamed tde_heap_basic. Use this access method name to create tables. Find guidelines in Test TDE tutorial.
The Beta version introduces the following bug fixes and improvements:
Fixed the issue with pg_tde running out of memory used for decrypted tuples. The fix introduces the new component TDEBufferHeapTupleTableSlot that keeps track of the allocated memory for decrypted tuples and frees this memory when the tuple slot is no longer needed.
Fixed the issue with adjusting a current position in a file by using raw file descriptor for the lseek function. (Thanks to user rainhard for providing the fix)
Enhanced the init script to consider a custom superuser for the POSTGRES_USER parameter when pg_tde is running via Docker (Thanks to Alejandro Paredero for reporting the issue)
The Alpha1 version of the extension introduces the following key features:
You can now rotate principal keys used for data encryption. This reduces the risk of long-term exposure to potential attacks and helps you comply with security standards such as GDPR, HIPAA, and PCI DSS.
You can now configure encryption differently for each database. For example, encrypt specific tables in some databases with different encryption keys while keeping others non-encrypted.
Keyring configuration has undergone several improvements, namely:
postgres-tde-ext to pg_tde. The extension name remains unchangedpg_tde sent many 404 requests to the Vault serverThe Minimum Viable Product (MVP) version introduces the following functionality:
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-,:!=\\[\\]()\"/]+|(?!\\b)(?=[A-Z][a-z])|\\.(?!\\d)|&[lg]t;","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"pg_tde documentation","text":"pg_tde is the open source PostgreSQL extension that provides Transparent Data Encryption (TDE) to protect data at rest. This ensures that the data stored on disk is encrypted, and no one can read it without the proper encryption keys, even if they gain access to the physical storage media.
You can configure encryption differently for each database, encrypting specific tables in some databases with different encryption keys while keeping others unencrypted.
Lear more what is Transparent Data Encryption and why you need it.
Important
This extension is in the experimental phase and is under active development. It is not meant for production use yet.
What\u2019s new
"},{"location":"index.html#whats-encrypted","title":"What\u2019s encrypted:","text":"pg_tde encrypts the following:
Warning: Note that introducing encryption/decryption affects performance. Our benchmark tests show less than 10% performance overhead for most situations. However, in some specific applications such as those using JSONB operations, performance degradation might be higher.
"},{"location":"index.html#versions-and-supported-postgresql-deployments","title":"Versions and supported PostgreSQL deployments","text":"The pg_tde extension comes in two distinct versions with specific access methods to encrypt the data. These versions are database-specific and differ in terms of what they encrypt and with what access method. Each version is characterized by the database it supports, the access method it provides, and the scope of encryption it offers.
Version for Percona Server for PostgreSQL
This pg_tde version is based on and supported for Percona Server for PostgreSQL 17.x - an open source binary drop-in replacement for PostgreSQL Community. It provides the tde_heap access method and offers full encryption capabilities.
Community version
This version is supported for PostgreSQL Community 16 and 17, and Percona Distribution for PostgreSQL 16. It provides the tde_heap_basic access method, offering limited encryption features. The limitations are in encrypting WAL data only for tables created using the extension and no support of index encryption nor logical replication.
The answer is pretty straightforward: for data sets where indexing is not mandatory or index encryption is not required, use the community version and the tde_heap_basic access method. Check the upstream documentation how to get started.
Otherwise, enjoy full encryption with the Percona Server for PostgreSQL version and the tde_heap access method.
Still not sure? Contact our experts to find the best solution for you.
Get started
"},{"location":"index.html#future-releases","title":"Future releases","text":"The following is planned for future releases of pg_tde:
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"apt.html","title":"Installpg_tde on Debian or Ubuntu","text":"The packages for the tech preview pg_tde are available in the experimental repository for Percona Distribution for PostgreSQL 17.
Check the list of supported platforms.
This tutorial shows how to install pg_tde with Percona Distribution for PostgreSQL.
You need the percona-release repository management tool that enables the desired Percona repository for you.
You need the following dependencies to install percona-release:
wgetgnupg2curllsb-releaseInstall them with the following command:
sudo apt-get install -y wget gnupg2 curl lsb-release\nFetch the percona-release package
sudo wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb\nInstall percona-release
sudo dpkg -i percona-release_latest.generic_all.deb\nEnable the Percona Distribution for PostgreSQL repository
Percona provides two repositories for Percona Distribution for PostgreSQL. We recommend enabling the Major release repository to timely receive the latest updates. Since the tde_heap access method is still in the experimental stage, the pg_tde package is currently available from the experimental repository.
$ sudo percona-release enable ppg-17.0 experimental\nUpdate the local cache
sudo apt-get update\npg_tde","text":"After all preconditions are met, install the extension.
Install Percona Distribution for PostgreSQL.
Run the following command to install Percona Distribution for PostgreSQL and the required packages:
sudo apt-get install -y percona-postgresql-17 percona-postgresql-contrib percona-postgresql-server-dev-all\nInstall pg_tde packages
sudo apt-get install percona-postgresql-17-pg-tde\nSetup
"},{"location":"apt.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"contribute.html","title":"Contributing guide","text":"Welcome to pg_tde - the Transparent Data Encryption extension for PostgreSQL!
We\u2019re glad that you would like to become a community member and contribute to this project.
You can contribute in one of the following ways:
Before submitting code contributions, we ask you to complete the following prerequisites.
"},{"location":"contribute.html#1-sign-the-cla","title":"1. Sign the CLA","text":"Before you can contribute, we kindly ask you to sign our Contributor License Agreement (CLA). You can do this in on click using your GitHub account.
Note: You can sign it later, when submitting your first pull request. The CLA assistant validates the PR and asks you to sign the CLA to proceed.
"},{"location":"contribute.html#2-code-of-conduct","title":"2. Code of Conduct","text":"Please make sure to read and agree to our Code of Conduct.
"},{"location":"contribute.html#submitting-a-pull-request","title":"Submitting a pull request","text":"All bug reports, enhancements and feature requests are tracked in GitHub issues. Though not mandatory, we encourage you to first check for a bug report among the issues and in the PR list: perhaps the bug has already been addressed.
For feature requests and enhancements, we do ask you to create a GitHub issue, describe your idea and discuss the design with us. This way we align your ideas with our vision for the product development.
If the bug hasn\u2019t been reported / addressed, or we\u2019ve agreed on the enhancement implementation with you, do the following:
Make your changes. Please follow the following guidelines to improve code readability:
Build pg_tde and test your changes locally.
When you work, you should periodically run tests to check that your changes don\u2019t break existing code.
To run the tests, use the following command:
cd pg_tde\nmake USE_PGXS=1 installcheck\nYou can run tests on your local machine with whatever operating system you have. After you submit the pull request, we will check your patch on multiple operating systems.
"},{"location":"contribute.html#contribute-to-documentation","title":"Contribute to documentation","text":"pg_tde documentation is written in Markdown language, so you can edit it online via GitHub. If you wish to have more control over the doc process, jump to how to edit documentation locally.
Before you start, learn what git, MkDocs and Docker are and what Markdown is and how to write it. For your convenience, there\u2019s also a cheat sheet to help you with the syntax.
The doc files are in the docs directory.
.md file of the page opens in GitHub editor in your browser. If you haven\u2019t worked with the repository before, GitHub creates a fork of it for you.This option is for users who prefer to work from their computer and / or have the full control over the documentation process.
The steps are the following:
git clone git@github.com:<your-name>/pg_tde.git\n\n3. Change the directory to ``pg_tde`` and add the remote upstream repository:\n\n```sh\ngit remote add upstream git@github.com:percona/pg_tde.git\ngit fetch upstream\ngit merge upstream/main\ngit checkout -b <my_branch>\nCommit your changes. The commit message guidelines will help you with writing great commit messages
Open a pull request to Percona
To verify how your changes look, generate the static site with the documentation. This process is called building. You can do it in these ways: - Use Docker - Install MkDocs and build locally
"},{"location":"contribute.html#use-docker","title":"Use Docker","text":"cd documentation\ndocker run --rm -v $(pwd):/docs perconalab/pmm-doc-md mkdocs build\nsite directory and open the index.html file to see the documentation.If you want to see the changes as you edit the docs, use this command instead:
cd documentation\ndocker run --rm -v $(pwd):/docs -p 8000:8000 perconalab/pmm-doc-md mkdocs serve --dev-addr=0.0.0.0:8000\nWait until you see INFO - Start detecting changes, then enter 0.0.0.0:8000 in the browser\u2019s address bar. The documentation automatically reloads after you save the changes in source files.
Install Python.
Install MkDocs and required extensions:
pip install -r requirements.txt\nBuild the site:
cd documentation\nmkdocs build\nOpen site/index.html
Or, to run the built-in web server:
cd documentation\nmkdocs serve\nView the site at http://0.0.0.0:8000
"},{"location":"contribute.html#build-pdf-file","title":"Build PDF file","text":"To build a PDF version of the documentation, do the following:
Disable displaying the last modification of the page:
export ENABLED_GIT_REVISION_DATE=false\nBuild the PDF file:
ENABLE_PDF_EXPORT=1 mkdocs build -f mkdocs-pdf.yml\nThe PDF document is in the site/pdf folder.
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"decrypt.html","title":"Decrypt an encrypted table","text":""},{"location":"decrypt.html#method-1-change-the-access-method","title":"Method 1. Change the access method","text":"If you encrypted a table with the tde_heap or tde_heap_basic access method and need to decrypt it, run the following command against the desired table (mytable in the example below):
ALTER TABLE mytable SET access method heap;\nCheck that the table is not encrypted:
SELECT pg_tde_is_encrypted('mytable');\nThe output returns f meaning that the table is no longer encrypted.
In the same way you can re-encrypt the data with the tde_heap_basic access method.
ALTER TABLE mytable SET access method tde_heap_basic;\nNote that the indexes and WAL files will no longer be encrypted.
"},{"location":"decrypt.html#method-2-create-a-new-unencrypted-table-on-the-base-of-the-encrypted-one","title":"Method 2. Create a new unencrypted table on the base of the encrypted one","text":"Alternatively, you can create a new unencrypted table with the same structure and data as the initial table. For example, the original encrypted table is EncryptedCustomers. Use the following command to create a new table Customers:
CREATE TABLE Customers AS\nSELECT * FROM EncryptedCustomers;\nThe new table Customers inherits the structure and the data from EncryptedCustomers.
(Optional) If you no longer need the EncryptedCustomers table, you can delete it.
DROP TABLE EncryptedCustomers;\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"external-parameters.html","title":"Use external reference to parameters","text":"To allow storing secrets or any other parameters in a more secure, external location, pg_tde allows users to specify an external reference instead of hardcoded parameters.
In Alpha1 version, pg_tde supports the following external storage methods:
file, which just stores the data in a simple file specified by a path. The file should be readable to the postgres process.remote, which uses a HTTP request to retrieve the parameter from the specified url.To use the file provider with a file location specified by the remote method, use the following command:
SELECT pg_tde_add_key_provider_file(\n 'file-provider', \n json_object( 'type' VALUE 'remote', 'url' VALUE 'http://localhost:8888/hello' )\n );\"\nOr to use the file method, use the following command:
SELECT pg_tde_add_key_provider_file(\n 'file-provider', \n json_object( 'type' VALUE 'remote', 'path' VALUE '/tmp/datafile-location' )\n );\"\nAny parameter specified to the add_key_provider function can be a json_object instead of the string, similar to the above examples.
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"faq.html","title":"FAQ","text":""},{"location":"faq.html#why-do-i-need-tde","title":"Why do I need TDE?","text":"Encrypting a hard drive encrypts all data including system and application files that are there. However, disk encryption doesn\u2019t protect your data after the boot-up of your system. During runtime, the files are decrypted with disk-encryption.
TDE focuses specifically on data files and offers a more granular control over encrypted data. It also ensures that files are encrypted on disk during runtime and when moved to another system or storage.
Consider using TDE and storage-level encryption together to add another layer of data security
"},{"location":"faq.html#is-tde-enough-to-ensure-data-security","title":"Is TDE enough to ensure data security?","text":"No. TDE is an additional layer to ensure data security. It protects data at rest. Consider introducing also these measures:
If you lose encryption keys, especially, the principal key, the data is lost. That\u2019s why it\u2019s critical to back up your encryption keys securely.
"},{"location":"faq.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"functions.html","title":"Functions","text":"The pg_tde extension provides the following functions:
Creates a new key provider for the database using a local file.
This function is intended for development, and stores the keys unencrypted in the specified data file.
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/keyring/data.file');\nAll parameters can be either strings, or JSON objects referencing remote parameters.
"},{"location":"functions.html#pg_tde_add_key_provider_vault_v2","title":"pg_tde_add_key_provider_vault_v2","text":"Creates a new key provider for the database using a remote HashiCorp Vault server.
The specified access parameters require permission to read and write keys at the location.
SELECT pg_tde_add_key_provider_vault_v2('provider-name',:'secret_token','url','mount','ca_path');\nwhere:
url is the URL of the Vault servermount is the mount point where the keyring should store the keyssecret_token is an access token with read and write access to the above mount pointca_path is the path of the CA file used for SSL verificationAll parameters can be either strings, or JSON objects referencing remote parameters.
"},{"location":"functions.html#pg_tde_set_principal_key","title":"pg_tde_set_principal_key","text":"Sets the principal key for the database using the specified key provider.
The principal key name is also used for constructing the name in the provider, for example on the remote Vault server.
You can use this function only to a principal key. For changes in the principal key, use the pg_tde_rotate_principal_key function.
SELECT pg_tde_set_principal_key('name-of-the-principal-key', 'provider-name');\nCreates a new version of the specified principal key and updates the database so that it uses the new principal key version.
When used without any parameters, the function will just create a new version of the current database principal key, using the same provider:
SELECT pg_tde_rotate_principal_key();\nAlternatively, you can pass two parameters to the function, specifying both a new key name and a new provider name:
SELECT pg_tde_rotate_principal_key('name-of-the-new-principal-key', 'name-of-the-new-provider');\nBoth parameters support the NULL value, which means that the parameter won\u2019t be changed:
-- creates new principal key on the same provider as before\nSELECT pg_tde_rotate_principal_key('name-of-the-new-principal-key', NULL);\n\n-- copies the current principal key to a new provider\nSELECT pg_tde_rotate_principal_key(NULL, 'name-of-the-new-provider');\nTells if a table is encrypted using the tde_heap access method or not.
To verify a table encryption, run the following statement:
SELECT pg_tde_is_encrypted('table_name');\nYou can also verify if the table in a custom schema is encrypted. Pass teh schema name for the function as follows:
SELECT pg_tde_is_encrypted('schema.table_name');\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"install.html","title":"Installation","text":""},{"location":"install.html#considerations","title":"Considerations","text":"You can use the following options to manage encryption keys:
Use the HashiCorp Vault server. This is the recommended approach. The Vault server configuration is out of scope of this document. We assume that you have the Vault server up and running. For the pg_tde configuration, you need the following information:
Use the local keyfile. This approach is rather used for development and testing purposes since the keys are stored unencrypted in the specified keyfile.
Install pg_tde using one of available installation methods:
The packages are available for the following operating systems:
Install on Debian or Ubuntu Install on RHEL or derivatives
To build pg_tde from source code, do the following
On Ubuntu/Debian: Install the following dependencies required for the build:
sudo apt install make gcc postgresql-server-dev-17 libcurl4-openssl-dev\nInstall Percona Distribution for PostgreSQL 17 or upstream PostgreSQL 17
If PostgreSQL is installed in a non standard directory, set the PG_CONFIG environment variable to point to the pg_config executable.
Clone the repository:
git clone git://github.com/percona/pg_tde\nCompile and install the extension
cd pg_tde\nmake USE_PGXS=1\nsudo make USE_PGXS=1 install\nNote
The steps below are for the pg_tde community version.
You can find Docker images built from the current main branch on Docker Hub. Images are built on top of postgres:17 official image.
To run pg_tde in Docker, use the following command:
docker run --name pg-tde -e POSTGRES_PASSWORD=mysecretpassword -d perconalab/pg_tde\nIt builds and adds pg_tde extension to PostgreSQL 17. The postgresql.conf contains the required modifications. The pg_tde extension is added to template1 so that all new databases automatically have the pg_tde extension loaded.
Keys are not created automatically. You must configure a key provider and a principal key for each database where you wish to use encrypted tables. See the instructions in the Setup section, starting with the 4th point, as the first 3 steps are already completed in the Docker image.
See Docker Docs on usage.
You can also build a Docker image manually with:
docker build . -f ./docker/Dockerfile -t your-image-name\nSetup
"},{"location":"install.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"setup.html","title":"Set uppg_tde","text":""},{"location":"setup.html#enable-extension","title":"Enable extension","text":"Load the pg_tde at the start time. The extension requires additional shared memory; therefore, add the pg_tde value for the shared_preload_libraries parameter and restart the postgresql instance.
Use the ALTER SYSTEM command from psql terminal to modify the shared_preload_libraries parameter.
ALTER SYSTEM SET shared_preload_libraries = 'pg_tde';\nStart or restart the postgresql instance to apply the changes.
$ sudo systemctl restart postgresql.service\n$ sudo systemctl restart postgresql-17\nCreate the extension using the CREATE EXTENSION command. You must have the privileges of a superuser or a database owner to use this command. Connect to psql as a superuser for a database and run the following command:
CREATE EXTENSION pg_tde;\nBy default, the pg_tde extension is created for the currently used database. To enable data encryption in other databases, you must explicitly run the CREATE EXTENSION command against them.
Tip
You can have the pg_tde extension automatically enabled for every newly created database. Modify the template template1 database as follows:
psql -d template1 -c 'CREATE EXTENSION pg_tde;'\nSet up a key provider for the database where you have enabled the extension.
With HashiCorp VaultWith keyring fileThe Vault server setup is out of scope of this document.
SELECT pg_tde_add_key_provider_vault_v2('provider-name',:'secret_token','url','mount','ca_path');\nwhere:
url is the URL of the Vault servermount is the mount point where the keyring should store the keyssecret_token is an access token with read and write access to the above mount pointca_path is the path of the CA file used for SSL verificationThis setup is intended for development and stores the keys unencrypted in the specified data file.
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/keyring/data.file');\nWarning: This example is for testing purposes only:
SELECT pg_tde_add_key_provider_file('file-vault','/tmp/pg_tde_test_local_keyring.per');\nAdd a principal key
SELECT pg_tde_set_principal_key('name-of-the-principal-key', 'provider-name');\nWarning: This example is for testing purposes only:
SELECT pg_tde_set_principal_key('test-db-master-key','file-vault');\nThe key is auto-generated.
Info: The key provider configuration is stored in the database catalog in an unencrypted table. See how to use external reference to parameters to add an extra security layer to your setup.
"},{"location":"setup.html#wal-encryption-configuration-tech-preview","title":"WAL encryption configuration (tech preview)","text":"After you enabled pg_tde and started the Percona Server for PostgreSQL, a principal key and internal keys for WAL encryption are created. They are stored in the data directory so that after WAL encryption is enabled, any process that requires access to WAL (a recovery or a checkpointer) can use them for decryption.
Now you need to instruct pg_tde to encrypt WAL files by configuring WAL encryption. Here\u2019s how to do it:
Use the ALTER SYSTEM SET command. You need the privileges of the superuser to run this command:
ALTER SYSTEM set pg_tde.wal_encrypt = on;\nRestart the server to apply the changes.
$ sudo systemctl restart postgresql.service\n$ sudo systemctl restart postgresql-17\nOn the server start
"},{"location":"setup.html#rotate-the-principal-key","title":"Rotate the principal key","text":"We highly recommend you to create your own keyring and rotate the principal key. This is because the default principal key is created from the local keyfile and is stored unencrypted.
Rotating the principal key means re-encrypting internal keys used for WAL encryption with the new principal key. This process doesn\u2019t stop the database operation meaning that reads and writes can take place as usual during key rotation.
Set up the key provider for WAL encryption
With HashiCorp VaultWith keyring fileSELECT pg_tde_add_key_provider_vault_v2('PG_TDE_GLOBAL','provider-name',:'secret_token','url','mount','ca_path');\nwhere:
PG_TDE_GLOBAL is the constant that defines the WAL encryption key provider-name is the name you define for the key providerurl is the URL of the Vault servermount is the mount point where the keyring should store the keyssecret_token is an access token with read and write access to the above mount pointca_path is the path of the CA file used for SSL verificationThis setup is intended for development and stores the keys unencrypted in the specified data file.
SELECT pg_tde_add_key_provider_file('provider-name','/path/to/the/keyring/data.file');\nRotate the principal key. Don\u2019t forget to specify the PG_TDE_GLOBAL constant to rotate only the principal key for WAL.
SELECT pg_tde_rotate_principal_key('PG_TDE_GLOBAL', 'new-principal-key', 'provider-name');\nNow all WAL files are encrypted for both encrypted and unencrypted tables.
Verify the encryption by checking the pg_tde.wal_encrypt GUC (Grand Unified Configuration) parameter as follows:
SELECT name, setting FROM pg_settings WHERE name = 'pg_tde.wal_encrypt';\n name | setting\n--------------------+---------\n pg_tde.wal_encrypt | on\nTest TDE
"},{"location":"setup.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"switch.html","title":"Switch from Percona Server for PostgreSQL to PostgreSQL Community","text":"Percona Server for PostgreSQL and PostgreSQL Community are binary compatible and enable you to switch from one to another. Here\u2019s how:
tde_heap (tech preview feature) access method for encryption, either re-encrypt the data using the tde_heap_basic access method, or decrypt it completely If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"table-access-method.html","title":"Table access method","text":"A table access method is the way how PostgreSQL stores the data in a table. The default table access method is heap. PostgreSQL organizes data in a heap structure, meaning there is no particular order to the rows in the table. Each row is stored independently and identified by its unique row identifier (TID).
heap access method works","text":"Insertion: When a new row is inserted, PostgreSQL finds a free space in the tablespace and stores the row there.
Deletion: When a row is deleted, PostgreSQL marks the space occupied by the row as free, but the data remains until it is overwritten by a new insertion.
Updates: PostgreSQL handles updates by deleting the old row and inserting a new row with the updated values.
"},{"location":"table-access-method.html#custom-access-method","title":"Custom access method","text":"Custom access methods allow you to implement and define your own way of organizing data in PostgreSQL. This is useful if the default table access method doesn\u2019t meet your needs.
Custom access methods are typically available with PostgreSQL extensions. When you install an extension and enable it in PostgreSQL, a custom access method is created.
An example of such an approach is the tde_heap access method. It is automatically created only for the databases where you enabled the pg_tde extension and configured the key provider, enabling you to encrypt the data.
To use a custom access method, specify the USING clause for the CREATE TABLE command:
CREATE TABLE table_name (\n column1 data_type,\n column2 data_type,\n ...\n) USING tde_heap;\ntde_heap works","text":"The tde_heap access method works on top of the default heap access method and is a marker to point which tables require encryption. It uses the custom storage manager TDE SMGR, which becomes active only after you installed the pg_tde extension.
Every data modification operation is first sent to the Buffer Manager, which updates the buffer cache. Then, it is passed to the storage manager, which then writes it to disk. When a table requires encryption, the data is sent to the TDE storage manager, where it is encrypted before written to disk.
Similarly, when a client queries the database, the PostgreSQL core sends the request to the Buffer Manager which checks if the requested data is already in the buffer cache. If it\u2019s not there, the Buffer Manager requests the data from the storage manager. The TDE storage manager reads the encrypted data from disk, decrypts it and loads it to the buffer cache. The Buffer Manager sends the requested data to the PostgreSQL core and then to the client.
Thus, the encryption is done at the storage manager level.
"},{"location":"table-access-method.html#changing-the-default-table-access-method","title":"Changing the default table access method","text":"You can change the default table access method so that every table in the entire database cluster is created using the custom access method. For example, you can enable data encryption by default by defining the tde_heap as the default table access method.
However, consider the following before making this change:
CREATE statement will default to the specified table access method. pg_tde extension and configure the key provider for all databases before you modify the configuration. Otherwise PostgreSQL won\u2019t find the specified access method and will throw an error.Here\u2019s how you can set the new default table access method:
Add the access method to the default_table_access_method parameter.
Use the ALTER SYSTEM SET command. This requires superuser or ALTER SYSTEM privileges.
This example shows how to set the tde_heap access method. Replace it with the tde_heap_basic if needed.
ALTER SYSTEM SET default_table_access_method=tde_heap;\nEdit the postgresql.conf configuration file and add the value for the default_table_access_method parameter.
This example shows how to set the tde_heap access method. Replace it with the tde_heap_basic if needed.
default_table_access_method = 'tde_heap'\nYou can use the SET command to change the default table access method temporarily, for the current session.
Unlike modifying the postgresql.conf file or using the ALTER SYSTEM SET command, the changes you make via the SET command don\u2019t persist after the session ends.
You also don\u2019t need to have the superuser privileges to run the SET command.
You can run the SET command anytime during the session. This example shows how to set the tde_heap access method. Replace it with the tde_heap_basic if needed.
SET default_table_access_method = tde_heap;\nReload the configuration to apply the changes:
SELECT pg_reload_conf();\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"tde.html","title":"What is Transparent Data Encryption (TDE)","text":"Transparent Data Encryption is a technology to protect data at rest. The encryption process happens transparently in the background, without affecting database operations. Data is automatically encrypted as it\u2019s written to the disk and decrypted as it\u2019s read, all in real-time. Users and applications interact with the data as usual without noticing any difference.
"},{"location":"tde.html#how-does-it-work","title":"How does it work?","text":"To encrypt the data, two types of keys are used:
pg_tde is integrated with HashiCorp Vault server to store and manage principal keys. Only the back end KV Secrets Engine - Version 2 (API) is supported.
The encryption process is the following:
When a user creates an encrypted table using pg_tde, a new random key is generated for that table using the AES128 (AES-ECB) cipher algorithm. This key is used to encrypt all data the user inserts in that table. Eventually the encrypted data gets stored in the underlying storage.
The table itself is encrypted using the principal key. The principal key is stored externally in the Vault key management store.
Similarly when the user queries the encrypted table, the principal key is retrieved from the key store to decrypt the table. Then the same unique internal key for that table is used to decrypt the data, and unencrypted data gets returned to the user. So, effectively, every TDE table has a unique key, and each table key is encrypted using the principal key.
"},{"location":"tde.html#why-do-you-need-tde","title":"Why do you need TDE?","text":"Using TDE has the following benefits:
For organizations:
For DBAs:
See also
Percona Blog: Transparent Data Encryption (TDE)
"},{"location":"tde.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"test.html","title":"Test Transparent Data Encryption","text":"Enabling pg_tde extension for a database creates the table access method tde_heap . This access method enables you to encrypt the data.
Here\u2019s how to do it:
Create a table in the database for which you have enabled pg_tde using the tde_heap access method as follows:
CREATE TABLE <table_name> (<field> <datatype>) USING tde_heap;\nWarning: Example for testing purposes only:
CREATE TABLE albums (\nalbum_id INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY,\nartist_id INTEGER,\ntitle TEXT NOT NULL,\nreleased DATE NOT NULL\n) USING tde_heap;\nLearn more about table access methods and how you can enable data encryption by default in the Table access methods section.
To check if the data is encrypted, run the following function:
SELECT pg_tde_is_encrypted('table_name');\nThe function returns t if the table is encrypted and f - if not.
Rotate the principal key when needed:
SELECT pg_tde_rotate_principal_key(); -- uses automatic key versionin\n-- or\nSELECT pg_tde_rotate_principal_key('new-principal-key', NULL); -- specify new key name\n-- or\nSELECT pg_tde_rotate_principal_key('new-principal-key', 'new-provider'); -- changeprovider\nYou can encrypt an existing table. It requires rewriting the table, so for large tables, it might take a considerable amount of time.
ALTER TABLE table_name SET access method tde_heap;\nHint
If you no longer wish to use pg_tde or wish to switch to using the tde_heap_basic access method, see how you can decrypt your data.
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"uninstall.html","title":"Uninstallpg_tde","text":"If you no longer wish to use TDE in your deployment, you can remove the pg_tde extension. To do that, your user must have the privileges of the superuser or a database owner.
Here\u2019s how to do it:
DROP EXTENSION with CASCADE command. Warning: The use of the CASCADE parameter deletes all tables that were created in the database with pg_tde enabled and also all dependencies upon the encrypted table (e.g. foreign keys in a non-encrypted table used in the encrypted one).
DROP EXTENSION pg_tde CASCADE\nRun the DROP EXTENSION command against every database where you have enabled the pg_tde extension
Modify the shared_preload_libraries and remove the \u2018pg_tde\u2019 from it. Use the ALTER SYSTEM SET command for this purpose
Start or restart the postgre instance to apply the changes.
sudo systemctl restart postgre.service\nsudo systemctl restart postgre-17\nIf you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"yum.html","title":"Installpg_tde on Red Hat Enterprise Linux and derivatives","text":"The packages for the tech preview pg_tde are available in the experimental repository for Percona Distribution for PostgreSQL 17.
Check the list of supported platforms.
This tutorial shows how to install pg_tde with Percona Distribution for PostgreSQL.
For RHEL8/Oracle Linux 8/Rocky Linux 8, disable the postgresql and llvm-toolsetmodules:
sudo dnf module disable postgresql llvm-toolset\npercona-release","text":"You need the percona-release repository management tool that enables the desired Percona repository for you.
Install percona-release:
sudo yum -y install https://repo.percona.com/yum/percona-release-latest.noarch.rpm \nEnable the repository
sudo percona-release enable-only ppg-17.0 experimental\nYou need to install the percona-postgresql{{ pgversion }}-devel package when working with pg_tde. This package requires dependencies that are not part of the Distribution, but can be installed from the specific repositories:
```\n$ sudo dnf config-manager --set-enabled ol9_codeready_builder\n```\npercona-postgresql{{ pgversion }}-devel package","text":""},{"location":"yum.html#install-pg_tde","title":"Install pg_tde","text":"Install Percona Distribution for PostgreSQL 17 and the required packages, run the following command:
sudo yum -y install percona-postgresql-client-common percona-postgresql-common percona-postgresql-server-dev-all percona-postgresql17 percona-postgresql17-contrib percona-postgresql17-devel percona-postgresql17-libs\nInstall pg_tde packages
sudo yum install percona-pg_tde_17\nSetup
"},{"location":"yum.html#get-expert-help","title":"Get expert help","text":"If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"},{"location":"release-notes/release-notes.html","title":"pg_tde release notes","text":"pg_tde extension brings in Transparent Data Encryption (TDE) to PostgreSQL and enables you to keep sensitive data safe and secure.
Get started
"},{"location":"release-notes/release-notes.html#beta-2024-06-30","title":"Beta (2024-06-30)","text":"With this version, the access method for pg_tde extension is renamed tde_heap_basic. Use this access method name to create tables. Find guidelines in Test TDE tutorial.
The Beta version introduces the following bug fixes and improvements:
Fixed the issue with pg_tde running out of memory used for decrypted tuples. The fix introduces the new component TDEBufferHeapTupleTableSlot that keeps track of the allocated memory for decrypted tuples and frees this memory when the tuple slot is no longer needed.
Fixed the issue with adjusting a current position in a file by using raw file descriptor for the lseek function. (Thanks to user rainhard for providing the fix)
Enhanced the init script to consider a custom superuser for the POSTGRES_USER parameter when pg_tde is running via Docker (Thanks to Alejandro Paredero for reporting the issue)
The Alpha1 version of the extension introduces the following key features:
You can now rotate principal keys used for data encryption. This reduces the risk of long-term exposure to potential attacks and helps you comply with security standards such as GDPR, HIPAA, and PCI DSS.
You can now configure encryption differently for each database. For example, encrypt specific tables in some databases with different encryption keys while keeping others non-encrypted.
Keyring configuration has undergone several improvements, namely:
postgres-tde-ext to pg_tde. The extension name remains unchangedpg_tde sent many 404 requests to the Vault serverThe Minimum Viable Product (MVP) version introduces the following functionality:
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.
Community Forum Get a Percona Expert
"}]} \ No newline at end of file diff --git a/main/sitemap.xml.gz b/main/sitemap.xml.gz index bb76055c..eef7c5ca 100644 Binary files a/main/sitemap.xml.gz and b/main/sitemap.xml.gz differ