Dump & restore
There are multiple ways to upgrade PostgreSQL version, but while the easier is to do a pg_dump from the OLD version and pg_restore in the NEW version, it's also the most time expensive (we're talking ~3 hours to dump and ~3 hours to restore in mid 2022).
Logical replica
Since PostgreSQL >= 10, logical replication has been natively implemented so it's possible to create a logical replication of the full database between different versions.
Official upgrade tool
The official pg_upgrade is designed to bump between different major versions, check cluster state, and other useful things.
This tool requires binaries and data folders to be available for both the OLD and NEW versions, and that's difficult if you're using containers. Fortunately, someone solved this problem creating a repository for Docker images with this precise scope.
The command to run the tool is:
docker run --rm -v "/PATH/TO/OLD":/var/lib/postgresql -e PGDATAOLD=/var/lib/postgresql/all/data -e PGDATANEW=/var/lib/postgresql/all/dataNEW tianon/postgres-upgrade:OLD-to-NEW --link
Where /PATH/TO/OLD contains the folder data for the OLD version.
Example output for an upgrade from 9.6 to 14, database size ~1.2TB, it took < 1 minute:
root@machine / # docker run -v "/PATH/TO/OLD":/var/lib/postgresql/all -e PGDATAOLD=/var/lib/postgresql/all/data -e PGDATANEW=/var/lib/postgresql/all/data14 tianon/postgres-upgrade:9.6-to-14 --link
Performing Consistency Checks
-----------------------------
Checking cluster versions ok
Checking database user is the install user ok
Checking database connection settings ok
Checking for prepared transactions ok
Checking for system-defined composite types in user tables ok
Checking for reg* data types in user tables ok
Checking for contrib/isn with bigint-passing mismatch ok
Checking for user-defined encoding conversions ok
Checking for user-defined postfix operators ok
Checking for incompatible polymorphic functions ok
Checking for tables WITH OIDS ok
Checking for invalid "sql_identifier" user columns ok
Checking for invalid "unknown" user columns ok
Creating dump of global objects ok
Creating dump of database schemas
postgres
template1
uala_development
ok
Checking for presence of required libraries ok
Checking database user is the install user ok
Checking for prepared transactions ok
Checking for new cluster tablespace directories ok
If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.
Performing Upgrade
------------------
Analyzing all rows in the new cluster ok
Freezing all rows in the new cluster ok
Deleting files from new pg_xact ok
Copying old pg_clog to new server ok
Setting oldest XID for new cluster ok
Setting next transaction ID and epoch for new cluster ok
Deleting files from new pg_multixact/offsets ok
Copying old pg_multixact/offsets to new server ok
Deleting files from new pg_multixact/members ok
Copying old pg_multixact/members to new server ok
Setting next multixact ID and offset for new cluster ok
Resetting WAL archives ok
Setting frozenxid and minmxid counters in new cluster ok
Restoring global objects in the new cluster ok
Restoring database schemas in the new cluster
template1
postgres
uala_development
ok
Adding ".old" suffix to old global/pg_control ok
If you want to start the old cluster, you will need to remove
the ".old" suffix from /var/lib/postgresql/all/data/global/pg_control.old.
Because "link" mode was used, the old cluster cannot be safely
started once the new cluster has been started.
Linking user relation files
/var/lib/postgresql/all/data/base/12409/2613
...
ok
Setting next OID for new cluster ok
Sync data directory to disk ok
Creating script to delete old cluster ok
Checking for hash indexes ok
Checking for extension updates notice
Your installation contains extensions that should be updated
with the ALTER EXTENSION command. The file
update_extensions.sql
when executed by psql by the database superuser will update
these extensions.
Upgrade Complete
----------------
Optimizer statistics are not transferred by pg_upgrade.
Once you start the new server, consider running:
/usr/lib/postgresql/14/bin/vacuumdb --all --analyze-in-stages
Running this script will delete the old cluster's data files:
./delete_old_cluster.sh
Notes
Using with bitnami PostgreSQL images
Bitnami's PostgreSQL images stores configuration outside data folder, while the official images do it, so in order to make this process work, it's needed to copy both postgresql.conf and pg_hba.conf from Bitnami's conf directory (/bitnami/postgresql/conf) to data dir.
This can be done by launching cp /bitnami/postgresql/conf/* /bitnami/postgresql/data/ from inside the container.
Before starting, the script need to:
- have right permission to folders:
chown -R postgres:postgres all - have
trustinpg_hba.confentrylocal all all - all extensions needed to start OLD version (
apt update && apt install postgresql-9.6-pgaudit)
Mounting volumes
When using the image with the faster --link option, the tool will create hard links instead of copying files, but this won't work between different volumes, filing with the error:
could not create hard link between old and new data directories: Invalid cross-device link
In link mode the old and new data directories must be on the same file system.
Failure, exiting
To solve that, mount a single volume with 2 folders: data for the OLD version and dataNEW for the NEW version in /var/lib/postgresql/all. Then, use env variables to let the tool know where OLD and NEW data folders are: PGDATAOLD=/var/lib/postgresql/all/data -e PGDATANEW=/var/lib/postgresql/all/dataNEW.