@minikit/mongotools
v0.0.2
Published
This project provides 2 wrappers mongodump and mongorestore
Downloads
1
Readme
@minikit/mongotools
This project provides 2 wrappers :
- mongodump,
- mongorestore.
There is an autonomous feature called rotation that provide a backup file rotation mechanism
- remove N oldest deprecated backups.
Command line usage
Initial setup - first time only
# get source code
git clone https://github.com/minikit/mongotools.git
# install dependencies
npm install
#~ setup environment variables
cp env/initEnv.template.sh env/initEnv.dontpush.sh
# you must update env/initEnv.dontpush.sh
Set your preferences
# source your options
. ./env/initEnv.dontpush.sh
Basic feature
# create a mongo dump
node mt dump
# create a encrypted mongo dump
node mt dumpz
# list backups
node mt list
# restore a mongo local dump
# please note that mt restore use following options : dropBeforeRestore: true, deleteDumpAfterRestore: true
node mt restore backup/myDatabase__2020-11-08_150102.gz
# rotate backup files
node mt rotation
# Helper : show current options values
node mt options
Add in-line extra options
You could play with env default options plus in-line command extra options
# create dump of a given 'shippingprices' collection, provide a target backup name as '2023_01_shippingPrices.gz', and show mongodump command
MT_COLLECTION=shippingprices MT_FILENAME=2023_01_shippingPrices.gz MT_SHOW_COMMAND=true node mt dump
# show backup in list
node mt list
# using mongo: drop a given collection
mongo myDb --eval "db.shippingprices.drop()"
# restore collection
MSYS_NO_PATHCONV=1 MT_COLLECTION=shippingprices MT_SHOW_COMMAND=true node mt restore /backup/2023_01_shippingprices.gz
# Note that collection option will produce wildcard in nsInclude arg '--nsInclude myDb.*'
Library use
Install dependency
You have to import as dependency
npm install @minikit/mongotools
Define the requirements, example:
import { MongoTools, MTOptions, MTCommand } from "@minikit/mongotools";
var mongoTools = new MongoTools();
const mtOptions = {
db: 'myDb',
port: 17017,
path: '/opt/myapp/backups',
dropboxToken: process.env.MYAPP_DROPBOX_SECRET_TOKEN
};
List dumps
var promiseResult = mongoTools.list(mtOptions);
Dump
var promiseResult = mongoTools.mongodump(mtOptions);
Restore
var promiseResult = mongoTools.mongorestore(mtOptions);
Rotation
var promiseResult = mongoTools.rotation(mtOptions);
Mongo tools options
Each mongotools feature rely on Mongo tools options (aka. MTOption).
Options precedence is the following:
- take
options
attribute if set, - else take related environment variable if any,
- else take default value if any,
- else if it's a mandatory option, throw an error.
TIP: you could also show current options by doing:
console.log(new MTOptions());
shared options
These options are used by dump and restore.
Either uri
or host
/port
/db
:
| option | env | required | default value | description |
| ------ | ------------ | -------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| uri
| MT_MONGO_URI | true | (none) | mongodump uri, example mongodb+srv://granted-user:[email protected]/tMyDatababse
. You could omit database name to dump all databases. |
or
| option | env | required | default value | description |
| ---------- | ---------------- | -------- | ------------- | --------------------------------------------------------------------------------- |
| db
| MT_MONGO_DB | true | (none) | mongo database name. For dump only, you could set it to *
to dump all databases |
| host
| MT_MONGO_HOST | false | 127.0.0.1
| mongo database hostname |
| port
| MT_MONGO_PORT | false | 27017
| mongo database port |
| username
| MT_MONGO_USER | false | (none) | mongo database username |
| password
| MT_MONGO_PWD | false | (none) | mongo database password |
| authDb
| MT_MONGO_AUTH_DB | false | admin
| mongo auth database |
ssl options
Optional ssl related options
| option | env | required | default value | description |
| ------------------- | ----------------------------- | -------- | ------------- | ---------------------------------------------------------------- |
| ssl
| MT_MONGO_SSL | false | (none) | if "1" then add --ssl
option |
| sslCAFile
| MT_MONGO_SSL_CA_FILE | false | (none) | .pem file containing the root certificate chain |
| sslPEMKeyFile
| MT_MONGO_SSL_PEM_KEY_FILE | false | (none) | .pem file containing the certificate and key |
| sslPEMKeyPassword
| MT_MONGO_SSL_PEM_KEY_PASSWORD | false | (none) | password to decrypt the sslPEMKeyFile, if necessary |
| sslCRLFile
| MT_MONGO_SSL_CRL_FILE | false | (none) | pem file containing the certificate revocation list |
| sslFIPSMode
| MT_MONGO_SSL_FIPS | false | (none) | if "1" then use FIPS mode of the installed openssl library |
| tlsInsecure
| MT_MONGO_TLS_INSECURE | false | (none) | if "1" then bypass the validation for server's certificate chain |
mongodump options
| option | env | required | default value | description |
| ------------------------ | ---------------------- | -------- | ----------------------- | --------------------------------------------------------------------- |
| path
| MT_PATH | false | backup
| dump target directory, created if it doesn't exist |
| dumpCmd
| | false | mongodump
| mongodump binary |
| fileName
| MT_FILENAME | false | <dbName_date_time.gz>
| dump target filename |
| encrypt
| | false | false | encrypt the dump using secret |
| secret
| MT_SECRET | false | null | secret to use if encrypt is enabled |
| encryptSuffix
| | false | .enc
| encrypt file suffix |
| includeCollections
| | false | (none) | Deprecated - please use collection
|
| collection
| MT_COLLECTION | false | (none) | Collection to include, if not specified all collections are included |
| excludeCollections
| MT_EXCLUDE_COLLECTIONS | false | (none) | Collections to exclude, if not specified all collections are included |
| numParallelCollections
| | false | 4 | Number of collections mongodump should export in parallel. |
| viewsAsCollections
| | false | false | When specified, mongodump exports read-only views as collections. |
Simple example:
import { MongoTools, MTOptions, MTCommand } from "node-mongotools";
var mongoTools = new MongoTools();
mongoTools.mongodump({
db:'myDatabase',
path:'backup',
username:'root', password:'mypass', authDb:'admin'
})
.then((success) => console.info("success", success) )
.catch((err) => console.error("error", err) );
mongorestore options
| option | env | required | default value | description |
| ------------------------ | ------------ | -------- | -------------- | -------------------------------------------------------------- |
| dumpFile
| MT_DUMP_FILE | true | (none) | dump file to restore |
| restoreCmd
| | false | mongorestore
| mongorestore binary |
| dropBeforeRestore
| | false | false | set it to true
to append --drop
option |
| deleteDumpAfterRestore
| | false | false | set it to true
to remove restored backup file |
| decrypt
| | false | false | decrypt the dump using secret. Activated if suffix is detected |
| secret
| MT_SECRET | false | null | secret to use if decrypt is enabled |
Simple example:
import { MongoTools, MTOptions, MTCommand } from "node-mongotools";
var mongoTools = new MongoTools();
mongoTools.mongorestore({
dumpFile:'backup/myDatabase__2020-11-8_160011.gz',
username:'root', password:'mypass', authDb:'admin'
})
.then((success) => {
console.info("success", success.message);
if (success.stderr) {
console.info("stderr:\n", success.stderr);// mongorestore binary write details on stderr
}
})
.catch((err) => console.error("error", err) );
Rotation options
A safe time windows is defined by :
now - rotationWindowsDays day(s)
===>now
where backups can't be removed.
Backup out of safe time windows are called deprecated backup
.
rotationMinCount
: minimum deprecated backups to keep,rotationCleanCount
: number of (oldest) deprecated backups to delete.
| option | env | required | default value | description |
| --------------------- | ------------------------ | -------- | ------------- | ------------------------------------------------------ |
| rotationDryMode
| MT_ROTATION_DRY_MODE | false | false | dont do delete actions, just print it |
| rotationWindowsDays
| MT_ROTATION_WINDOWS_DAYS | true | 15 | safe time windows in days since now |
| rotationMinCount
| MT_ROTATION_MIN_COUNT | true | 2 | minimum deprecated backups to keep. |
| rotationCleanCount
| MT_ROTATION_CLEAN_COUNT | true | 10 | number of (oldest first) deprecated backups to delete. |
Simple example:
MT_ROTATION_CLEAN_COUNT=2 \
MT_ROTATION_DRY_MODE=true \
MT_ROTATION_WINDOWS_DAYS=3 \ node mt rotation
Example details: if there is a backup that is more than 3 days old, keep 2 newer ones and delete the 10 oldest.