npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

ci-pg-restore

v1.0.11

Published

A cached perf optimized restoration of a PostgreSQL database dump

Downloads

1,184

Readme

ci-pg-restore: A cached perf optimized restoration of a PostgreSQL database dump

usage: ci-pg-restore [-h] -d DBNAME --cache-name CACHE_NAME
                     [--deps-files "some/**/file1.sql\nfile2.sql\n..."]
                     [--deps-files-cmd "find . -name '*.sql'"]
                     [--cache-db-max-age 42, 42s, 30m, 8h, 3d, 1w]
                     [--restore-db-max-age 42, 42s, 30m, 8h, 3d, 1w]
                     ...

A cached perf optimized restoration of a PostgreSQL database dump.

To access the databases, the tool assumes that psql CLI is installed
in the system, and PGUSER/PGPASSWORD are correctly configured.

The tool uses the following observation: if we have a large rarely
changed *.sql file with a database dump, and we need to restore it
multiple times (in e.g. CI environment, when running integration
test matrix), it's way faster to restore it only once in some cache
"template" database and then run CREATE DATABASE WITH TEMPLATE to
clone that cache database multiple times.

While restoring, the tool behaves nicely in terms of concurrency: if
multiple processes of the tool are running at the same time, they
wait for each other.

The tool also takes care of cleaning up the cache databases and the
restored databases. This is typically useful in CI environment, when
the databases are not needed anymore after the tests are finished.

positional arguments:
  shell command         a shell command which will be run to restore
                        the database if its cache is invalid; the
                        database name will also be passed via
                        PGDATABASE environment variable; you may
                        call the regular psql here or use any other
                        database migration tool

options:
  -h, --help            show this help message and exit
  -d DBNAME, --dbname DBNAME
                        the name of the database to restore; it will
                        be pre-created empty prior to calling the
                        restoration command
  --cache-name CACHE_NAME
                        cache name; the tool will create an internal
                        cached "template" database suffixed with the
                        digest of that name plus any --deps-files or
                        --deps-files-cmd involved
  --deps-files "some/**/file1.sql\nfile2.sql\n..."
                        a newline separated list of file name
                        wildcards involved in the restoration (you
                        may use this arg multiple times; all files
                        must exist); the tool will create an
                        internal cached "template" database suffixed
                        with the digest of those files content
  --deps-files-cmd "find . -name '*.sql'"
                        a shell command which should return a
                        newline separated list of all file names
                        involved in the restoration; see --deps-
                        files for details
  --cache-db-max-age 42, 42s, 30m, 8h, 3d, 1w
                        if set, the cached "template" databases will
                        be deleted if they are not used for that
                        long
  --restore-db-max-age 42, 42s, 30m, 8h, 3d, 1w
                        if set, the restored databases will be
                        deleted if they are not used for that long