npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@zoff-tech/firestore-import-export

v1.1.1

Published

Firestore data import and export

Downloads

3

Readme

node-firestore-import-export

Firestore data importing, exporting, and data clearing tool.

THIS IS A FORK OF https://github.com/jloosli/node-firestore-import-export

codebeat badge Codacy Badge David badge Known Vulnerabilities CircleCI

Export a Firestore database, including collections and documents, while keeping the structure intact.

Table of Contents

Data Format

Exports a json file with the following format:

{
  "__collections__": {
    "companies": {
      "docA": {
        "name": "Big Co",
        "employee_count": 2012,
        "created": {
          "__datatype__": "timestamp",
          "value": {
            "_seconds": 12343456,
            "_nanoseconds": 7890
          }
        },
        "location": {
          "__datatype__": "geopoint",
          "value": {
            "_latitude": -123.456789,
            "_longitude": 34.5678
          }
        },
        "AdministratorRef": {
          "__datatype__": "documentReference",
          "value": "path/to/the/document"
        },
        "__collections__": {
          "employees": ...,
          "products": ...
        }
      },
      "docB": ...,
      "docC": ...
    },
    "administrators": {
      "docA": ...,
      "docB": ...
    }
  }
}

where __collections__ holds the collections for a given document (or the root of the database).

Imports need to be from a file with the same structure (e.g. from an exported file).

Be careful! This can easily overwrite or mess up your data if you import to the wrong location.

Special Datatypes

Three types of data are serialized in the export:

  • Timestamps
  • Geopoints
  • DocumentReferences

They each are serialized in the following format:

{
  "__datatype__": "timestamp|geopoint|documentReference",
  "value": "The serialized value"
}

Installation

Install using npm.

npm install -g node-firestore-import-export

or yarn

yarn global add node-firestore-import-export

Alternatively download the source.

git clone https://github.com/jloosli/node-firestore-import-export.git

Retrieving Google Cloud Account Credentials

  1. Visit the Firebase Console
  2. Select your project
  3. Navigate to Project Settings (at the time of writing the gear icon button at the top left of the page).
  4. Navigate to Service Accounts
  5. Click Generate New Private Key

This downloaded json file contains the proper credentials needed for node-firestore-import-export to authenticate.

Using Firebase Firestore Emulator

If using Firebase Emulators, all commands (Export, Import, and Clear) will override the account credentials setting if the FIRESTORE_EMULATOR_HOST environment variable is set.

Usage

Command Line

The path to the account credentials can either be passed with the -a/--accountCredentials flag, or placed in the GOOGLE_APPLICATION_CREDENTIALS environment variable. For example:

export GOOGLE_APPLICATION_CREDENTIALS=path/to/my/credentials.json
firestore-export -p

Export

  • -a, --accountCredentials <path> - path to Google Cloud account credentials JSON file. If missing, will look at the GOOGLE_APPLICATION_CREDENTIALS environment variable for the path.
  • -b, --backupFile <path>- Filename to store backup. (e.g. backups/full-backup.json). Defaults to firestore-export.json if missing.
  • -n, --nodePath <path>- Path to database node to start (e.g. collectionA/docB/collectionC). Backs up full database if empty or missing.
  • -p, --prettyPrint - JSON backups done with pretty-printing.
Examples
Export full database
firestore-export --accountCredentials path/to/credentials/file.json --backupFile /backups/myDatabase.json
Export with pretty printing
firestore-export --accountCredentials path/to/credentials/file.json --backupFile /backups/myDatabase.json --prettyPrint
Export from a specific path (and all its children/collections)
firestore-export --accountCredentials path/to/credentials/file.json --backupFile /backups/myDatabase.json --nodePath collectionA/document1/collectionCC

Import

  • -a, --accountCredentials <path> - path to Google Cloud account credentials JSON file. If missing, will look at the GOOGLE_APPLICATION_CREDENTIALS environment variable for the path.
  • -b, --backupFile <path>- Filename with backup data. (e.g. backups/full-backup.json).
  • -n, --nodePath <path>- Path to database node to start (e.g. collectionA/docB/collectionC).
  • -y, --yes - Unattended import without confirmation (like hitting "y" from the command line).
Examples
Import full database
firestore-import --accountCredentials path/to/credentials/file.json --backupFile /backups/myDatabase.json
Import to a specific path
firestore-import --accountCredentials path/to/credentials/file.json --backupFile /backups/myDatabase.json --nodePath collectionA/document1/collectionCC

Clear

  • -a, --accountCredentials <path> - path to Google Cloud account credentials JSON file. If missing, will look at the GOOGLE_APPLICATION_CREDENTIALS environment variable for the path.
  • -n, --nodePath <path>- Path to database node to start (e.g. collectionA/docB/collectionC).
  • -y, --yes - Unattended clear without confirmation (like hitting "y" from the command line). Command will wait 5 seconds so you can Ctrl-C to stop.
  • -w, --noWait - Combine this with the --yes confirmation to not wait 5 seconds
Example
Clear everything under a specific node
firestore-clear --accountCredentials path/to/credentials/file.json --yes

Library

The underlying library can be used in a node or web application for importing and exporting data in a similar fashion

Exporting

import {firestoreExport} from 'node-firestore-import-export';
import * as firebase from 'firebase-admin';

firebase.initializeApp({
    apiKey: "AIza....",                             
    authDomain: "YOUR_APP.firebaseapp.com",         
    databaseURL: "https://YOUR_APP.firebaseio.com", 
    storageBucket: "YOUR_APP.appspot.com",          
    messagingSenderId: "123456789"                  
});

const collectionRef = firebase.firestore().collection('collectionA/docB/collectionC');

firestoreExport(collectionRef)
    .then(data=>console.log(data));

Importing

import {firestoreImport} from 'node-firestore-import-export';
import * as firebase from 'firebase-admin';

firebase.initializeApp({
    apiKey: "AIza....",                             
    authDomain: "YOUR_APP.firebaseapp.com",         
    databaseURL: "https://YOUR_APP.firebaseio.com", 
    storageBucket: "YOUR_APP.appspot.com",          
    messagingSenderId: "123456789"                  
});

const data = {
  docA: {
    name: 'bob',
    __collections__: {}
  },
  docB: {
    name: 'jill',
    __collections__: {}
  }
};

const collectionRef = firebase.firestore().collection('collectionA/docB/collectionC');

firestoreImport(data, collectionRef)
    .then(()=>console.log('Data was imported.'));

Clearing

import {firestoreClear} from 'node-firestore-import-export';
import * as firebase from 'firebase-admin';

firebase.initializeApp({
    apiKey: "AIza....",                             
    authDomain: "YOUR_APP.firebaseapp.com",         
    databaseURL: "https://YOUR_APP.firebaseio.com", 
    storageBucket: "YOUR_APP.appspot.com",          
    messagingSenderId: "123456789"                  
});

const collectionRef = firebase.firestore().collection('collectionA/docB/collectionC');

firestoreClear(collectionRef)
    .then(()=>console.log('Everything under collectionA/docB/collectionC was removed.'));

Contributions

Feel free to report bugs and make feature requests in the Issue Tracker, fork and create pull requests!

Inspiration

The command line was inspired heavily by SteadyEquipment's node-firestore-backup

Support on Beerpay

Hey dude! Help me out for a couple of :beers:!

Beerpay Beerpay