A completely reworked fork of scottwrobinson's camo

Changes:

• Complete code refactoring & optimization
• allows using any fork of NeDB, like @justlep/nedb

  import {Datastore} from '@justlep/nedb';
  import {connect} from '@justlep/camo';
    
  await connect('nedb:///path/to/dbfiles', Datastore);
    
  // The Datastore argument is now mandatory.
  // For using the original '[email protected]', you would do
    
  import Datastore from 'nedb'; 
  await connect('nedb:///path/to/dbfiles', Datastore);

• analog to the "unique" flag in Document schemas (ensuring a unique index for a field), there is now an additional "indexed" flag ensuring a non-unique index for the respective field, e.g.

  class Book extends Document {
   static SCHEMA = {
     isbn: {
       type: String,
       unique: true        
     },
     year: {
       type: Number,
       indexed: true
     }
   }
  }

• After removing properties from a document schema, those now-void properties may remain in the db file entries since camo only updates values for schema properties. To ensure db files don't get messed up over time with obsolete properties, you should purge obsolete properties on application startup or similar:

  setUnknownDataKeyBehavior('logAndIgnore');
  // Regardless of the handler configured above, the unknown data key behavior is DISABLED during the purge 
  // (i.e. no logging, no throwing whatsoever ).
  // The behavior is restored to the above one after the purge.
    
  let {totalUpdates, ids} = await MyDocClass.purgeObsoleteProperties(['voidProp1', 'voidProp2'], true);
  // ids == array of changed documents' ids (or null if last argument is false)
  // The db file's current latest entries will be lacking the void properties.

  IMPORTANT: Due to the nature of nedb, the db files after purging will still contain both the old version AND the new, purged version of all updated documents until the next startup/compacting. If you purged sensitive data, you can force immediate compaction by adding a third argument:

  let {totalUpdates} = await MyDocClass.purgeObsoleteProperties(['voidProp1', 'voidProp2'], false, true);
  // The db file's current latest entries will be lacking the void properties, 
  // the db file is compacted

Breaking changes:

• Removed Mongo support for now, leaving NeDB only
• Removed nedb from optionalDependencies, must be installed manually:

  # for Node 14+/ESM (see code example above)
  npm i --save @justlep/nedb 
    
  # or the original but unmaintained NeDB v1.8.0
  npm i --save nedb

• Accessing id properties of Document or EmbeddedDocument no longer displays deprecation warnings, but will throw an Error.
• Passing a collection name to new MyDocument(collectionName) now throws an Error (must override static collectionName() instead)
• Documents now throw errors when created with data objects containing keys that are not present in the schema. This behavior can be customized.
• Schema definition by constructor (or instance.schema()) is deprecated. Use a static field SCHEMA instead, e.g.

  class Foo extends Document {
    static SCHEMA = {
      num: {
        type: Number,
        default: 123
      }
    }
  }
    
  Foo.create().num === 123; // true

  More info below at Declaring Your Document
• Client._dropDatabase() is now private/internal (see CHANGELOG)

Camo

Jump To

• Why do we need another ODM?
• Advantages
• Install and Run
• Quick Start
  • Connect to the Database
  • Declaring Your Document
    • Embedded Documents
  • Creating and Saving
  • Loading
    • Behavior on unknown data keys
  • Deleting
  • Counting
  • Hooks
  • Misc.
• Transpiler Support
• Contributing
• Contact
• Copyright & License

Why do we need another ODM?

Short answer, we probably don't. Camo was created for two reasons: ~~to bring traditional-style classes to MongoDB JavaScript, and~~ to support NeDB as a backend (which is much like the SQLite-alternative to Mongo).

Throughout development this eventually turned in to a library full of ES6 features. Coming from a Java background, its easier for me to design and write code in terms of classes, and I suspect this is true for many JavaScript beginners. While ES6 classes don't bring any new functionality to the language, they certainly do make it much easier to jump in to OOP with JavaScript, which is reason enough to warrent a new library, IMO.

Advantages

So, why use Camo?

• ES6: ES6 features are quickly being added to Node, especially now that it has merged with io.js. With all of these new features being released, Camo is getting a head start in writing tested and proven ES6 code. This also means that native Promises are built-in to Camo, so no more promisify-ing your ODM or waiting for Promise support to be added natively.
• Easy to use: While JavaScript is a great language overall, it isn't always the easiest for beginners to pick up. Camo aims to ease that transition by providing familiar-looking classes and a simple interface. Also, there is no need to install a full MongoDB instance to get started thanks to the support of NeDB.
• Multiple backends: Camo was designed and built with multiple Mongo-like backends in mind, like NeDB, LokiJS*, and TaffyDB*. With NeDB support, for example, you don't need to install a full MongoDB instance for development or for smaller projects. This also allows you to use Camo in the browser, since databases like NeDB supports in-memory storage.
• Lightweight: Camo is just a very thin wrapper around the backend databases, which mean you won't be sacrificing performance.

* Support coming soon.

Install and Run

To use Camo, you must first have installed Node >=14, then run the following commands:

npm install @justlep/camo --save

And at least ONE of the following:

npm install @justlep/nedb --save

OR

npm install nedb --save

Quick Start

Camo was built with ease-of-use and ES6 in mind, so you might notice it has more of an OOP feel to it than many existing libraries and ODMs. Don't worry, focusing on object-oriented design doesn't mean we forgot about functional techniques or asynchronous programming. Promises are built-in to the API. Just about every call you make interacting with the database (find, save, delete, etc) will return a Promise. No more callback hell :)

For a short tutorial on using Camo, check out this article.

Connect to the Database

Before using any document methods, you must first connect to your underlying database. All supported databases have their own unique URI string used for connecting. The URI string usually describes the network location or file location of the database. However, some databases support more than just network or file locations. NeDB, for example, supports storing data in-memory, which can be specified to Camo via nedb://memory. See below for details:

• NeDB:
  • Format: "nedb://[directory-path]" OR "nedb://memory"
  • Example: const uri = 'nedb:///Users/scott/data/db-files';

So to connect to an NeDB database, use the following:

import {Datastore} from '@justlep/nedb';
import {connect} from '@justlep/camo';

const database = await await connect('nedb:///path/to/dbfiles', Datastore);
  
// The Datastore argument is now mandatory.
// For using the original '[email protected]', you would do
  
import Datastore from 'nedb'; 
const database = await connect('nedb:///path/to/dbfiles', Datastore);

Declaring Your Document

All models must inherit from the Document class, which handles much of the interface to your backend NoSQL database.

import {Document} from '@justlep/camo';

class Company extends Document {
    static SCHEMA = {
        name: String,
        valuation: {
            type: Number,
            default: 10000000000,
            min: 0
        },
        employees: [String],
        dateFounded: {
            type: Date,
            default: Date.now
        }
    };

    static collectionName() {
        return 'companies';
    }
}

The schema is defined by a static SCHEMA property. Schemas are analyzed once per class at the moment the first instance of the class (or a derived class) gets instantiated. Changing SCHEMA thereafter will have no effect.

The name of the collection can be set by overriding the static collectionName() method, which should return the desired collection name as a string. If one isn't given, then Camo uses the name of the class and naively appends an 's' to the end to make it plural.

The SCHEMA property can also be a function returning a schema object. This allows for circular document-type references. For example:

class Foo extends Document {
    static SCHEMA = () => ({bar: Bar});
}

class Bar extends Document {
    static SCHEMA = {foo: Foo};
}

Supported variable types are:

• String
• Number
• Boolean
• Buffer
• Date
• (Object)
• (Array)
• EmbeddedDocument
• Document Reference

Arrays either can be declared as either un-typed (using Array or []), or typed (using the [TYPE] syntax, like [String]). Typed arrays are enforced by Camo on .save() and an Error will be thrown if a value of the wrong type is saved in the array. Arrays of references are also supported.

Untyped arrays and "wildcard" objects

Untyped arrays and Object-type fields require additional schema properties fromData, toData and validate for sanitizing raw data to be stored to or restored from the database, and for validating the value before save():

Example:

// the 'obj' property of these instances are saved as strings in the db 
class CustomObjectModel extends Document {
    static SCHEMA = {
        obj: {
            type: Object,
            toData: JSON.stringify, // converts the current value into a format the db can save  
            fromData: JSON.parse,   // parses the value coming from the database  
            validate: (o) => o && typeof o === 'object' && o.hi === 'bye'
        }
    };
}

Declaring a member variable in the schema

To declare a member variable in the schema, either directly assign it one of the types listed above, or assign it an object with options, like this:

class Foo extends Document {
    static SCHEMA = {
        anyNumber: Number,
        primeNumber: {
            type: Number,
            default: 2,
            min: 0,
            max: 25,
            choices: [2, 3, 5, 7, 11, 13, 17, 19, 23],
            unique: true
        }
    };
}

The default option supports both values and no-argument functions (like Date.now). Currently the supported options/validators are:

• type: The value's type (required)
• default: The value to be assigned if none is provided (optional)
• min: The minimum value a Number can be (optional)
• max: The maximum value a Number can be (optional)
• choices: A list of possible values (optional)
• match: A regex string that should match the value (optional)
• validate: A 1-argument function that returns false if the value is invalid (optional)
• unique: A boolean value indicating if a 'unique' index should be set (optional)
• required: A boolean value indicating if a key value is required (optional)

To reference another document, just use its class name as the type.

class Dog extends Document {
    static SCHEMA = {
        name: String,
        breed: String
    };  
}

class Person extends Document {
    static SCHEMA = {
        pet: Dog,
        name: String,
        age: Number
    };

    static collectionName() {
        return 'people';
    }
}

Embedded Documents

Embedded documents can also be used within Documents. You must declare them separately from the main Document that it is being used in. EmbeddedDocuments are good for when you need an Object, but also need enforced schemas, validation, defaults, hooks, and member functions. All of the options (type, default, min, etc) mentioned above work on EmbeddedDocuments as well.

import {Document, EmbeddedDocument} from '@justlep/camo';

class Money extends EmbeddedDocument {
    static SCHEMA = {
        value: {
            type: Number,
            choices: [1, 5, 10, 20, 50, 100]
        },
        currency: {
            type: String,
            default: 'usd'
        }
    };
}

class Wallet extends Document {
    static SCHEMA = {
        contents: [Money]
    };
}

let wallet = Wallet.create();
wallet.contents.push(Money.create());
wallet.contents[0].value = 5;
wallet.contents.push(Money.create());
wallet.contents[1].value = 100;

await wallet.save();
console.log('Both Wallet and Money objects were saved!');

Creating and Saving

To create a new instance of our document, we need to use the .create() method, which handles all of the construction for us.

let lassie = Dog.create({
    name: 'Lassie',
    breed: 'Collie'
});

lassie.save().then(function(l) {
    console.log(l._id);
});

Once a document is saved, it will automatically be assigned a unique identifier by the backend database. This ID can be accessed by the ._id property.

If you specified a default value (or function) for a schema variable, that value will be assigned on creation of the object.

An alternative to .save() is .findOneAndUpdate(query, update, options). This static method will find and update (or insert) a document in one atomic operation (atomicity is guaranteed in MongoDB only). Using the {upsert: true} option will return a new document if one is not found with the given query.

Loading

Both the find and delete methods following closely (but not always exactly) to the MongoDB API, so it should feel fairly familiar.

If querying an object by id, you must use _id and not id.

To retrieve an object, you have a few methods available to you.

• .findOne(query, options) (static method)
• .find(query, options) (static method)

The .findOne() method will return the first document found, even if multiple documents match the query. .find() will return all documents matching the query. Each should be called as static methods on the document type you want to load.

let lassie = await Dog.findOne({ name: 'Lassie' });
console.log('Got Lassie!');
console.log('Her unique ID is', lassie._id);

.findOne() currently accepts the following option:

• populate: Boolean value to load all or no references. Pass an array of field names to only populate the specified references
  • Person.findOne({name: 'Billy'}, {populate: true}) populates all references in Person object
  • Person.findOne({name: 'Billy'}, {populate: ['address', 'spouse']}) populates only 'address' and 'spouse' in Person object

.find() currently accepts the following options:

• populate: Boolean value to load all or no references. Pass an array of field names to only populate the specified references
  • Person.find({lastName: 'Smith'}, {populate: true}) populates all references in Person object
  • Person.find({lastName: 'Smith'}, {populate: ['address', 'spouse']}) populates only 'address' and 'spouse' in Person object
• sort: Sort the documents by the given field(s)
  • Person.find({}, {sort: '-age'}) sorts by age in descending order
  • Person.find({}, {sort: ['age', 'name']}) sorts by ascending age and then name, alphabetically
• limit: Limits the number of documents returned
  • Person.find({}, {limit: 5}) returns a maximum of 5 Person objects
• skip: Skips the given number of documents and returns the rest
  • Person.find({}, {skip: 5}) skips the first 5 Person objects and returns all others

Behavior on unknown data keys

Refactorings in collections or Document classes may, over time, lead to NeDB data objects containing "old" properties which are no longer part of a document's schema. Since it can't be considered safe in all cases to silently ignore or accept such "unknown data keys", the default behavior of @justlep/camo is now to throw an Error.

This behavior can be configured globally by calling setUnknownDataKeyBehavior():

import {setUnknownDataKeyBehavior} from '@justlep/camo';

setUnknownDataKeyBehavior('throw');  // throw with error message containg key + class name
setUnknownDataKeyBehavior('ignore'); // silently ignore unknown data
setUnknownDataKeyBehavior('accept'); // silently accept 
setUnknownDataKeyBehavior('logAndAccept'); // accept and log key + class name to the console 
setUnknownDataKeyBehavior('logAndIgnore'); // ignore and log key + class name to the console
setUnknownDataKeyBehavior('default'); // switch back to default (=throw)  

// or set global custom handlers (this-context will be the instance the data shall be assigned to) 
setUnknownDataKeyBehavior(function(dataKey, dataVal) {
    console.warn('unknonwn key %s for class %s', dataKey. this.constructor.name);
    if (dataKey[0] !== '_') {
        this[dataKey] = dataVal;
    }
});

Alternatively, you can override onUnknownData() for individual classes:

class Foo extends Document {
  static SCHEMA = {};
  
  /** @override */
  onUnknownData(dataKey, dataVal) {
      this[dataKey] = dataVal; // silently accept unkown keys
  }
}

let foo = Foo.create({xxx: 666}); // won't throw
foo.xxx === 666; // true

Note: only document properties mentioned in the schema will be persisted, regardless of the settings above. Accepting random data keys does not mean those properties will be saved or updated in the database.

Deleting

To remove documents from the database, use one of the following:

• .delete()
• .deleteOne(query, options) (static method)
• .deleteMany(query, options) (static method)
• .findOneAndDelete(query, options) (static method)

The .delete() method should only be used on an instantiated document with a valid id. The other three methods should be used on the class of the document(s) you want to delete.

let numDeleted = await Dog.deleteMany({ breed: 'Collie' });
console.log(`Deleted ${numDeleted} Collies from the database.`);

Counting

To get the number of matching documents for a query without actually retrieving all of the data, use the .count() method.

let totalCollies = await Dog.count({ breed: 'Collie' });
console.log(`Found ${totalCollies} Collies`);

Hooks

Camo provides hooks for you to execute code before and after critical parts of your database interactions. For each hook you use, you may return a value (which, as of now, will be discarded) or a Promise for executing asynchronous code. Using Promises throughout Camo allows us to not have to provide separate async and sync hooks, thus making your code simpler and easier to understand.

Hooks can be used not only on Document objects, but EmbeddedDocument objects as well. The embedded object's hooks will be called when it's parent Document is saved/validated/deleted (depending on the hook you provide).

In order to create a hook, you must override a class method. The hooks currently provided, and their corresponding methods, are:

• pre-validate: preValidate()
• post-validate: postValidate()
• pre-save: preSave()
• post-save: postSave()
• pre-delete: preDelete()
• post-delete: postDelete()

Here is an example of using a hook (pre-delete, in this case):

class Company extends Document {
    static SCHEMA = {
        employees: [Person]
    };

    preDelete() {
        return Promise.all(this.employees.map(emp => emp.delete()));
    }
}

The code above shows a pre-delete hook that deletes all the employees of the company before it itself is deleted. As you can see, this is much more convenient than needing to always remember to delete referenced employees in the application code.

[!] Note: The .preDelete() and .postDelete() hooks are only called when calling .delete() on a Document instance. Calling .deleteOne() or .deleteMany() will not trigger the hook methods.

Misc.

• camo.getClient(): Retrieves the Camo database client
• camo.getClient().driver(): Retrieves the underlying database driver (MongoClient or a map of NeDB collections)
• Document.toJSON(): Serializes the given document to just the data, which includes nested and referenced data

Transpiler Support

(This information may be obsolete)

While many transpilers won't have any problem with Camo, some need extra resources/plugins to work correctly:

• Babel
  • babel-preset-camo: Babel preset for all es2015 plugins supported by Camo
• TypeScript
  • DefinitelyTyped/camo: Camo declaration file (h/t lucasmciruzzi)
  • IndefinitivelyTyped/camo: Typings support for Camo (h/t WorldMaker)

Contributing

Feel free to open new issues or submit pull requests for Camo.

https://github.com/justlep/camo/issues

Before opening an issue or submitting a PR, I ask that you follow these guidelines:

Issues

• Please state whether your issue is a question, feature request, or bug report.
• Always try the latest version of Camo before opening an issue.
• If the issue is a bug, be sure to clearly state your problem, what you expected to happen, and what all you have tried to resolve it.
• Always try to post simplified code that shows the problem. Use Gists for longer examples.

Pull Requests

• If your PR is a new feature, please consult with me first.
• Any PR should contain only one feature or bug fix. If you have more than one, please submit them as separate PRs.
• Always try to include relevant tests with your PRs. If you aren't sure where a test should go or how to create one, feel free to ask.
• Include updates to the README when needed.
• Do not update the package version or CHANGELOG. I'll handle that for each release.

Copyright & License

Copyright (c) 2022 Lennart Pegel

Copyright (c) 2016 Scott Robinson

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.