ci-pg-restore
v1.0.11
Published
A cached perf optimized restoration of a PostgreSQL database dump
Downloads
1,184
Maintainers
Readme
ci-pg-restore: A cached perf optimized restoration of a PostgreSQL database dump
usage: ci-pg-restore [-h] -d DBNAME --cache-name CACHE_NAME
[--deps-files "some/**/file1.sql\nfile2.sql\n..."]
[--deps-files-cmd "find . -name '*.sql'"]
[--cache-db-max-age 42, 42s, 30m, 8h, 3d, 1w]
[--restore-db-max-age 42, 42s, 30m, 8h, 3d, 1w]
...
A cached perf optimized restoration of a PostgreSQL database dump.
To access the databases, the tool assumes that psql CLI is installed
in the system, and PGUSER/PGPASSWORD are correctly configured.
The tool uses the following observation: if we have a large rarely
changed *.sql file with a database dump, and we need to restore it
multiple times (in e.g. CI environment, when running integration
test matrix), it's way faster to restore it only once in some cache
"template" database and then run CREATE DATABASE WITH TEMPLATE to
clone that cache database multiple times.
While restoring, the tool behaves nicely in terms of concurrency: if
multiple processes of the tool are running at the same time, they
wait for each other.
The tool also takes care of cleaning up the cache databases and the
restored databases. This is typically useful in CI environment, when
the databases are not needed anymore after the tests are finished.
positional arguments:
shell command a shell command which will be run to restore
the database if its cache is invalid; the
database name will also be passed via
PGDATABASE environment variable; you may
call the regular psql here or use any other
database migration tool
options:
-h, --help show this help message and exit
-d DBNAME, --dbname DBNAME
the name of the database to restore; it will
be pre-created empty prior to calling the
restoration command
--cache-name CACHE_NAME
cache name; the tool will create an internal
cached "template" database suffixed with the
digest of that name plus any --deps-files or
--deps-files-cmd involved
--deps-files "some/**/file1.sql\nfile2.sql\n..."
a newline separated list of file name
wildcards involved in the restoration (you
may use this arg multiple times; all files
must exist); the tool will create an
internal cached "template" database suffixed
with the digest of those files content
--deps-files-cmd "find . -name '*.sql'"
a shell command which should return a
newline separated list of all file names
involved in the restoration; see --deps-
files for details
--cache-db-max-age 42, 42s, 30m, 8h, 3d, 1w
if set, the cached "template" databases will
be deleted if they are not used for that
long
--restore-db-max-age 42, 42s, 30m, 8h, 3d, 1w
if set, the restored databases will be
deleted if they are not used for that long