o-snapshot
v2.0.0
Published
Objects serializer, for NodeJS and JS
Downloads
1
Maintainers
Readme
Snapshot
Objects serializer, for NodeJS and JS
Among other features, o-snapshot
preserves the identity of serialized objects, after deserialization
preserves the class of serialized objects, after deserialization
gives the programmer complete control over which classes can be serialized, and which classes can't
gives the programmer complete control over which attributes to serialize, and how
supports a simple mechanism to upgrade a snapshot from a previous version, saved using a different format
when used with classes, rather than plain Objects, the size of a serialized snapshot is an order of magnitude below the size of the same snapshot in plain json
supports circular references
supports the serialization of object surrogates
runs on both Node >= v10, and browsers >= ES9
Additionaly, used as an in-memory objects database, whether in browser or Node.js, o-snapshot
supports in-memory, atomic transactions
implements an extremely naive, and a bit of not entirely correct, garbage collection strategy
Installation
npm install o-snapshot
Documentation
Basic usage
o-snapshot
has 2 main concepts
First one, it's the Serializer
A Serializer
converts back and forth between instances of a single class, and json objects
Second one, it's the ObjectsDatabase
ObjectsDatabase
takes an object, finds a serializer for it, and serializes or deserializes it, along with the objects it references
InMemoryDatabase
is a concrete ObjectsDatabase
Example of use
You can browse the complete, functional example, in here
Say there are 2 classes
const { SequentialCollection } = require('o-toolbox')
class Book {
constructor({ author, title, isbn }) {
this.author = author
this.title = title
this.isbn = isbn
this.editions = new SequentialCollection()
}
addBookEdition({ bookEdition }) {
bookEdition._setBook({ book: this })
this.editions.add( bookEdition )
}
addAllBookEditions({ bookEditions }) {
bookEditions.forEach( (bookEdition) => {
this.addBookEdition({ bookEdition })
})
}
getAuthor() {
return this.author
}
getAllEditions() {
return this.editions.copy()
}
getIsbn() {
return this.isbn
}
getTitle() {
return this.title
}
}
module.exports = Book
and
const { SequentialCollection } = require('o-toolbox')
class BookEdition {
constructor({ publisher, dateOfPublishing, numberOfPages }) {
this.publisher = publisher
this.dateOfPublishing = dateOfPublishing
this.numberOfPages = numberOfPages
this.copies = new SequentialCollection()
this.book = null
}
addBookCopy({ bookCopy }) {
bookCopy._setBookEdition({ bookEdition: this })
this.copies.add( bookCopy )
}
addAllBookCopies({ bookCopies }) {
bookCopies.forEach( (bookCopy) => {
this.addBookCopy({ bookCopy })
})
}
getAuthor() {
return this.book.getAuthor()
}
_setBook({ book }) {
this.book = book
}
}
module.exports = BookEdition
Create a serializer for Book
objects
const Book = require('../models/Book')
class BookSerializer {
getClass () {
return Book
}
getClassId () {
return 'Book'
}
getClassVersion () {
return 1
}
serialize ( book ) {
const { title, author, isbn, editions } = book
return [ title, author, isbn, editions ]
}
deserialize ({ attributes }) {
const [ title, author, isbn, editions ] = attributes
const book = new Book({ title, author, isbn })
book.addAllBookEditions({ bookEditions: editions })
return book
}
}
module.exports = BookSerializer
and a serializer for BookEdition
objects
const BookEdition = require('../models/BookEdition')
class BookEditionSerializer {
getClass () {
return BookEdition
}
getClassId () {
return 'BookEdition'
}
getClassVersion () {
return 1
}
serialize ( bookEdition ) {
const { publisher, dateOfPublishing, numberOfPages, copies } = bookEdition
return [ publisher, dateOfPublishing, numberOfPages, copies ]
}
deserialize ({ attributes }) {
const [ publisher, dateOfPublishing, numberOfPages, copies ] = attributes
const bookEdition = new BookEdition({ publisher, dateOfPublishing, numberOfPages })
bookEdition.addAllBookCopies({ bookCopies: copies })
return bookEdition
}
}
module.exports = BookEditionSerializer
Serializers
A Serializer
can be any object, as long as it defines the methods
getClass
Returns the class of the objects it serializes
getClassId
Returns the name of the class it serializes, or any other unique id
In a regular context, it may seem redundant to have two different methods to get a class, and its name
There are use cases where it's needed to, though
Such a case is when the serializer is evaluated in an environment that has no access to class names, like a deployment with its source code minified or obfuscated. That's a common case in Single Page Applications
Another one, is when a refactor changes the class name, but the application should preserve backwards compatibility with objects already serialized
getClassVersion
Returns the current version of the class
The class version is an integer, greater or equal than 1
Later in this document, there's an example that shows what to use the class version for
serialize
Takes an instance of the class, and returns either an array, or a json object, with the attributes to serialize
serialize()
is a regular method, and can get the attributes using the object public interface. It does not need to know the object internals
It's possible to write serializers for objects defined in a third party library
serialize()
can add, change or remove any attribute of the object being serialized
The attributes returned by .serialize()
might include values, like Number and String, null, undefined, and instances of other classes
o-snapshot
takes care of serializing referred instances too, if they have a serializer
deserialize
It does the opposite to the serialize
method
It takes the serialized attributes, and creates and returns an instance of the class
The attributes it receives are already deserialized to their class instance form
deserialize()
also takes a second parameter, the serialized class version, to upgrade the serialized object if it needs to
InMemoryDatabase
The entry point to an object serialization, or deserialization, is an InMemoryDatabase
Before using it, configure it with the serializers of choice:
const { InMemoryDatabase } = require('o-snapshot')
const inMemoryDatabase = new InMemoryDatabase()
.register({ serializer: new BookSerializer() })
.register({ serializer: new BookEditionSerializer() })
.register({ serializer: new SequentialCollectionSerializer() })
Then, add an object, and take a snapshot
const book = new Book({
title: 'Hamlet',
author: 'William Shakespeare',
isbn: '112233'
})
const snapshot = inMemoryDatabase
.setAt({ key: 'root', value: book })
.takeSnapshot()
It's possible to serialize more than one root object in the same snapshot, if it makes sense for the program
const snapshot = inMemoryDatabase
.setAt({ key: 'users', value: users })
.setAt({ key: 'products', value: products })
.takeSnapshot()
To restore back a snapshot
inMemoryDatabase.restoreSnapshot({ snapshot })
const book = inMemoryDatabase.at({ key: 'root' })
How does it work
When InMemoryDatabase
adds a Book
object, it looks for the object class (specifically, it looks for the object.constructor
) in its registered serializers
If it finds one, it calls serializer.serialize( object )
, to get the list of the attributes to serialize
It does the same with each one of the returned attributes
Once it iterates through all the reference graph, starting at the given object, it flattens the objects graph to an array of json attributes
Objects identity
The identity of each object is preserved after its deserialization
Serialized objects don't need to implement an identity surrogate. It uses the identity of the object in memory
E.g.
Before the serialization of
const book = new Book({
title: 'Hamlet',
author: 'William Shakespeare',
isbn: '112233'
})
const books = [ book, book, book ]
Array books
contains 3 occurrences of the identical object. Each element in the array points to the same object, book
After its serialization
const { InMemoryDatabase } = require('o-snapshot')
const inMemoryDatabase = new InMemoryDatabase()
.register({ serializer: new BookSerializer() })
.register({ serializer: new BookEditionSerializer() })
.register({ serializer: new SequentialCollectionSerializer() })
const snapshot = inMemoryDatabase
.setAt({ key: 'root', value: books })
.takeSnapshot()
and deserialization
const { InMemoryDatabase } = require('o-snapshot')
const inMemoryDatabase = new InMemoryDatabase()
.register({ serializer: new BookSerializer() })
.register({ serializer: new BookEditionSerializer() })
.register({ serializer: new SequentialCollectionSerializer() })
inMemoryDatabase.restoreSnapshot({ snapshot })
const books = inMemoryDatabase.at({ key: 'root' })
the identity of the objects is preserved, and the new books
Array also contains 3 occurrences of a single object, rather than 3 equal objects with different identities
During the serialization of a complex graph of objects, to preserve the invariant of the identity of objects may be significant. Otherwise, the program may have unexpected errors, like an object not emitting events as expected
In addition, it can reduce significantly the size of a snapshot, if it holds many refences to identical objects
E.g, a snapshot of a collection of product orders
{
orders : [
{
user,
products: [
{ product }, ...
]
},
...
]
}
may have multiple references to identical products and users
Within the snapshot, each object is serialized, with its attributes, only once
References to other objects are serialized as ObjectReference
s instead, using an Integer
Built-in serializers
Although it's quite possible, and even quite simple, to make a serializer capable to serialize any object at all, rather than to require the programmer to declare each serializer explicitly, it proved to be a troubled design
To explicitly declare serializers makes the program more secure, more simple to debug, and more easy to understand
In addition, it allows to use different serializers, in different contexts, for the same classes, and features like class versioning, and surrogate references
Some built-in types come with serializers out of the box
Objects of type
- undefined
- null
- String
- Number
- Array
- Object
- Boolean
don't need to declare a custom serializer
Any other class, it does
Built-in serializers can be overriden, as well
Surrogate references
Say the application uses a global reference
// global variable
const localClock = new Clock()
user.setClock( clock )
const snapshot = inMemoryDatabase
.setAt({ key: 'root', value: user })
.takeSnapshot()
The restoration of the snapshot taken wouldn't preserve the identity of localClock
object
It would create a new instance of Clock
, different from the localClock
To preserve the reference to localClock
object, the serializer must replace the reference by a surrogate object, and restore it back, at the time of its deserialization
const User = require('./User')
class UserSerializer {
getClass() {
return User
}
getClassId() {
return 'User'
}
getClassVersion() {
return 1
}
serialize(user) {
const clock = 'localClock'
return [ clock ]
}
deserialize({ attributes, classVersion }) {
const [ clock ] = attributes
const user = new User()
user.setClock( global.localClock )
return user
}
}
module.exports = UserSerializer
The same technique can be applied to serialize references to resources that may not persist between the time of object serialization, and the time of object deserialization, such as database connections, file handles, network sockets or GUI elements
E.g.
const User = require('./User')
class UserSerializer {
constructor({ connectionPool }) {
this.connectionPool = connectionPool
}
getClass() {
return User
}
getClassId() {
return 'User'
}
getClassVersion() {
return 1
}
serialize( user ) {
const name = user.getName()
const databaseConfig = user.getDatabase().getConfig()
return [ name, databaseConfig ]
}
deserialize({ attributes, classVersion }) {
const [ name, databaseConfig ] = attributes
const user = new User({ name })
const databaseConnection = this.connectionPool.getDatabaseConnection( databaseConfig )
user.setDatabase(databaseConnection)
return user
}
}
module.exports = UserSerializer
Class versions and migrations
Once an application persists at least one object, a programmer needs to address a whole new set of problems
Say there's a class
class User {
constructor({ lastName }) {
this.lastName = lastName
}
...
}
with its serializer
const User = require('./User')
class UserSerializer {
getClass() {
return User
}
getClassId() {
return 'User'
}
getClassVersion() {
return 1
}
serialize( user ) {
const lastName = user.lastName
return { lastName }
}
deserialize({ attributes, classVersion }) {
const { lastName } = attributes
return new User({ lastName })
}
}
module.exports = UserSerializer
A snapshot is taken at time t1
const snapshotAtTime1 = inMemoryDatabase
.setAt({ key: 'root', value: user })
.takeSnapshot()
then, at time t2, a refactor is made to User
class
class User {
constructor({ lastname }) {
this.lastname = lastname
}
...
}
Even it may not seem like that much of a change, the rename of the instance variable will make any snapshot of the User
class taken before t2, to fail the next time it's restored
Issues appear because the application, or the environment where the application runs, may change between the time at which the snapshot is taken, and the time at which the snapshot is restored
Serializers use the getClassVersion
method to deal with this kind of errors
To properly handle the upgrade, every time User
class changes its attributes, the serializer also upgrades it's version number
const { ImportMethods } = require('o-toolbox')
const { SerializerVersionUpgrades } = require('o-snapshot')
const User = require('./User')
class UserSerializer {
getClass() {
return User
}
getClassId() {
return 'User'
}
getClassVersion() {
return 2 // <-- increased by 1
}
serialize( user ) {
const lastname = user.lastname
return { lastname }
}
deserialize({ attributes, classVersion }) {
if ( classVersion === 1 ) {
const { lastName } = attributes
const lastname = lastName
return new User({ lastname })
}
const { lastname } = attributes
return new User({ lastname })
}
}
module.exports = UserSerializer
Note that getClassVersion
increments its value by 1
Validations
It's a good idea to validate each attribute before each object serialization/deserialization
It may save a lot of debugging time
o-snapshot
does not include any validation utility
Any assertions or validations library can be used within serialize
and deserialize
methods for that matter
The author of o-snapshot
, that would be me, recomends o-check-list, for it's simplicity, ease of use, and adaptability to different contexts, since it works in both Node.js >= 10, and browser regular JS >= ES9
DoMe commands
DoMe commands are intended to be self-documented, please take a look at the files in DoMe/forDevelopment/inWindows
, or in DoMe/forDevelopment/inDocker