goodies

Multiple MongoDB databases and cross-database sessions

MongoDB data modeling stands on Mongoose, so we have to use Mongoose's API to connect to MongoDB. Mongoose allows us to reuse one primary connection to leverage cross-database sessions. If we need this, we must define connectionDetails slightly differently, as the below example illustrates:

[
	{
		id: 'primary',
		type: 'mongo-db',
		connectionDetails: {
			host: '${gestalt.environment.DATABASE_HOST}',
			port: 27017,
			replicaSet: 'local'
		}
	},
	{
		id: 'secondary',
		type: 'mongo-db',
		connectionDetails: {
			reuseId: 'primary'
		}
	},
	{
		id: 'tertiary',
		type: 'mongo-db',
		connectionDetails: {
			reuseId: 'primary'
		}
	}
]

assets

As the above schema denotes, asset size limitations are determined in the following way:

1. Type minimum/maximum size.
2. If not present, general minimum/maximum size.
3. If not present, defaults: DEFAULT_MINIMUM/MAXIMUM_ASSET_SIZE.

Note: keep in mind that unless you absolutely know what you're doing, these should really never change, as it might result in inconsistent/invalid data.

Permissions

Permissions are enforced on data models via:

1. Optional getPermissionConditions(). If present, it'll be added as the last condition on every query.
2. Mandatory enforcePermissions. Must return true.

This mostly works. However, if we'd do this on every data model, it'd likely slow us down. To optimise, we expect the HTTP layer to enforce this. For example, consider the following schema:

Workspace
	-> Project (is a child of workspace)
		-> Document (is a child of project)

If we define permissions on workspace, there's no sense in enforcing them on project as well, because that:

• already implicitly applies with the parent-child relationship
• would be an unnecessary repetition

Mongoose

Only use save(). Never insertMany(), findOneAndUpdate(), etc. Reason is to have one hook, and essentially a simpler setup. Also not really possible to do the same things with certain calls.

Data Hooks

When creating or patching:

1. Data model onPreProcess post data entity initialization.
2. Data entity onValidate.
3. Data model onValidate.
4. Data model onPreSave. At this point, data has been succesfully validated, and is ready to be persisted. Mutations are no longer allowed, and will simply be ignored. Barring unforeseen circumstances, persistence should succeed. Use this for external actions, like for example, sending emails.
5. Data model onPostSave. Same as above.

When deleting:

1. Data model onPreDelete.
2. Data model onPostDelete.

Mongoose Chained Reference Query Language (CRQL)

CRQL was developed to be used internally for the purpose of easily resolving chained references. Initially it was developed to serve goodiesForeignConstraints, but was later generalised and is now being used in multiple places.

We always start with 1 to any amount of context documents which we call zeroth documents. We can take any path on these documents and traverse through an unlimited amount of references through any amount of data models to reach the final destination:

AssociatedDataModel1(zerothPath).path->AssociatedDataModel2(firstPath).path->AssociatedDataModel3(secondPath).path->third.path

Or a little bit easier to read:

AssociatedDataModel1(zerothPath).path
->AssociatedDataModel2(firstPath).path
->AssociatedDataModel3(secondPath).path
->third.path

For example, consider we have a document with projectId. This document also contains a value with userId. We can take userId and reference it via a project and workspace user roles, to make sure the document belongs to a project that belongs to a workspace that this user can access:

Project(@projectId)._id
->Workspace(@workspaceId)._id
->userRoles.userId

Note that at (@) references global/top level fields in case of sub documents. In the final step, we can reference multiple paths by separating them with a comma (,):

Project(@projectId)._id
->Workspace(@workspaceId)._id
->userRoles.userId,something.else,and.also.this

A few things to consider:

1. This saves us a ton of custom logic. So if possible, always go with CRQL.
2. There's no limitation as to how deep the connection can go.
3. There's no formatting limitations. In other words, use any amount of white spaces you need to make it more readable.
4. The only thing to be careful about really, is to always query indexed fields to keep everything as performant as possible.

Mongoose Data Schema Plugins

goodiesForeignConstraints

Note: this is only implemented on the top level schema to optimise the amount of queries. Duplicates are ignored.

Checks for reference existence. Doesn't include deleted documents.

There are two seemingly different types, which work the same way under the hood.

1. Direct

Used to check one model. It requires a model and one or more paths:

Model.path1[,path2,path3]

For example, if we want to reference a user, we can do:

User._id

We're not limited to primary keys. We can use any path:

User.email

Mind the word path, meaning we can also do:

User.path.to.a.nested.field

Or multiple paths:

User.path.to.a.nested.field,path.to.another.nested.field,path.to.yet.another.nested.field

A few things to consider:

1. This saves us a ton of custom logic. So if possible, always go with this.

2. There's no formatting limitations. In other words, use any amount of white spaces you need to make it more readable.

3. The only thing to be careful about really, is to always query indexed fields to keep everything as performant as possible.

4. Indirect

See CRQL.

goodiesLimitByReference

Note: this is only implemented on the top level schema to optimise the amount of queries.

Limits the field value(s) by reference. Consider the following schema:

{
	confirmedMemberUserIds: {
		type: [
			uuid
		]
	},
	invitedEmails: {
		type: [
			string
		],
		goodiesLimitByReference: 'User(confirmedMemberUserIds)._id->email'
	}
}

This is a relatively complicated scenario. On one hand, we have user IDs, and on the other, emails. Things that are impossible to compare directly. With the given query of User(confirmedMemberUserIds)._id->email, we can check if within confirmedMemberUserIds already exists a user with one of the emails provided in invitedEmails, and prevent it. This way we cannot invite a user that is already a member.

goodiesUniqueArraySubDocument

Checks for uniqueness of the given keys. Doesn't include deleted records.

goodiesLock

Locks the given keys. Includes deleted documents.

Have a nice day 😘