nest-query-parser
v1.4.3
Published
A query string parser to be used in applications developed with NestJS.
Downloads
27
Readme
NOTE: This library will no longer have updates. To use the same existing features, and take advantage of new features and eventual fixes, use the nest-mongo-query-parser library. For applications that use Typeorm, I'm developing a library called nest-typeorm-query-parser. Happy Coding! :)
Summary
- Prerequisites
- Installing
- Usage
- Examples
- Explain the Resources
- Rules
- Observations
- Practical Examples
- Upcoming Features
- License
- Authors
Prerequisites
As the name of the library suggests, this library was built to work together with the NestJS framework. However, as future work, another library will be implemented that can be used as middleware of APIs that use Express or HapiJS.
Installing
Use the follow command:
npm i --save nest-query-parser
Usage
There are two ways to use the parsers available in this library: as a ParamDecorator or as a MethodDecorator.
If you want to use it as a ParamDecorator, just add the tag referring to the Parser to be used as a method parameter. Example:
import { Get } from '@nestjs/common';
import { Controller } from '@nestjs/common';
import { ResourceService } from './resource.service';
import { MongoQuery, MongoQueryModel } from 'nest-query-parser';
@Controller('resources')
export class ResourceController {
constructor(private readonly _service: ResourceService) {}
@Get()
public find(@MongoQuery() query: MongoQueryModel) {
return this._service.find(query);
}
}
It can also be used as a MethodDecorator. Just use the tag referring to the Parser to be used as the method decorator. Example:
import { Injectable } from '@nestjs/common';
import { MongoQueryParser, MongoQueryModel } from 'nest-query-parser';
@Injectable()
export class ResourceService {
@MongoQueryParser()
public find(query: MongoQueryModel) {
return [];
}
}
NOTE: When using the library as a MethodDecorator, you can receive other arguments in the method in question, but the query has to be passed as the first argument of the function, so that the treatment is done properly.
Examples
Request: http://localhost:3000/resources
Query:
{
"limit": 100,
"skip": 0,
"select": {},
"sort": {},
"populate": [],
"filter": {}
}
Request: http://localhost:3000/resources?limit=10&page=2&select=_id,name,age&sort=-created_at&age=gt:30
Query:
{
"limit": 10,
"skip": 10,
"select": {
"_id": 1,
"name": 1,
"age": 1
},
"sort": {
"created_at": -1
},
"populate": [],
"filter": {
"age": {
"$gt": 30
}
}
}
Explain the Resources
Queries with @MongoQuery() | @MongoQueryParser()
Pagination
The paging feature is very useful for clients who will consume your API. It is through this feature that applications can define the data limit in a query, as well as define which page to be displayed. Each time a page of an application is selected, it means that some resources have been displaced (data offset or skip data).
There is a mathematical rule that relates page number to resource offset. Basically:
offset = (page - 1) * limit, where page > 0.
This means that for a limit of 10 elements per page:
- To access page 1, the offset will be equal to = (1 - 1) * 10, so offset = 0
- To access page 2, the offset will be equal to = (2 - 1) * 10, so offset = 10
- To access page 3, the offset will be equal to = (3 - 1) * 10, so offset = 20
And so on.
With this library, it is possible to use pagination with the page
parameter, or using the skip
manually. By default,
the limit
value is 100
and skip
value is 0
.
Example:
Request: http://localhost:3000/resources?limit=10&page=3
Query:
{
"limit": 10,
"skip": 20
}
Request: http://localhost:3000/resources?limit=10&skip=20
Query:
{
"limit": 10,
"skip": 20
}
Ordering
To work with ordering, you need to specify one or more sorting parameters, and whether you want the sorting to be ascending or descending. For ascending ordering, just put the name of the ordering parameter. For descending ordering, you need to put a "-" symbol before the name of the ordering parameter. Example:
Request: http://localhost:3000/resources?sort=created_at
Query:
{
"sort": {
"created_at": 1
}
}
Request: http://localhost:3000/resources?sort=-created_at
Query:
{
"sort": {
"created_at": -1
}
}
Request: http://localhost:3000/resources?sort=-age,name
Query:
{
"sort": {
"age": -1,
"name": 1
}
}
In multiple-parameter ordering, the first ordering parameter has higher priority than the second, and so on. In the
example above, the ordering will be given primarily by the age
parameter, in descending order. If there are two or
more objects with the same value in age
, then those objects will be sorted by name
in ascending order.
Select
With this library, you can choose which parameters should be returned by the API. However, Mongo has a peculiarity: you can also specify which parameters you don't want to be returned. The logic is similar to ordering: to specify which parameters are to be returned, simply enter the parameter name; and to specify which parameters should not be returned, just place a "-" symbol before the parameter.
Example:
Request: http://localhost:3000/resources?select=_id,name,age
Query:
{
"select": {
"_id": 1,
"name": 1,
"age": 1
}
}
Request: http://localhost:3000/resources?select=-_id,-created_at,-updated_at
Query:
{
"select": {
"_id": 0,
"created_at": 0,
"updated_at": 0
}
}
It is interesting to use one or the other in your queries, as one is complementary to the other. If you want almost all parameters except a few, use the option to ignore parameters. If you want some parameters, and ignore the others, use the option to select the ones you want.
Filters
Now let's go to the most complex part of the library: the filters. There are several ways to apply filters in this library, so I'm going to break this topic down into subtopics for every possible filter approach.
Simple Filters
Simple filters are equality filters. Basically it's set key=value. All filter parameters are defined as string, so there are some validations that are done on these values.
If the value is a string number, it is transformed into a number, either integer or float/double (up to 16 decimal places);
If the value is in yyyy-MM-dd format or yyyy-MM-ddThh:mm:ss.sZ format, it is transformed into a Date object;
If the value is 'true' or 'false', it is transformed into a boolean value, according to your value;
Otherwise, the value is considered as a string.
Example:
Request: http://localhost:3000/resources?name=John%20Doe&age=31&birth_date=1990-01-01
Query:
{
"filter": {
"name": "John Doe",
"age": 31,
"birth_date": 1990-01-01T00:00:00.000Z
}
}
MultiLevel Filters
You can specify multilevel filters. This means that, if you have an object that has a field that is another object, you can perform a search with filters through the parameters of the internal object. Example:
Object
{
"_id": "613532a350857c1c8d1d10d9",
"name": "Filippo Nyles",
"age": 28,
"current_job": {
"title": "Budget/Accounting Analyst III",
"salary": 4776.8
}
}
Request: http://localhost:3000/resources?current_job.title=Budget/Accounting%20Analyst%20III
Query:
{
"filter": {
"current_job.title": "Budget/Accounting Analyst III"
}
}
Partial Filters
Partial filters are a way to search a string type value for a part of the value. There are three ways to use partial
filters. Making an analogy with javascript, it would be like using the startsWith
, endsWith
and includes
methods,
where:
- startsWith: search for a string-type value that starts with a given substring. To do this, just add a "*" at the beginning of the substring.
- endsWith: search for a string-type value that ends with a given substring. To do this, just add a "*" at the end of the substring.
- includes: search for a string value that contains a specific substring. To do this, just add a "*" at the beginning and end of the substring.
Example:
Request: http://localhost:3000/resources?name=Lu&email=gmail.com&job=Developer
Query:
{
"filter": {
"name": {
"$regex": "^Lu",
"$options": "i"
},
"email": {
"$regex": "gmail.com$",
"$options": "i"
},
"job": {
"$regex": "Developer",
"$options": "i"
}
}
}
Comparison Filters
Comparison operators are specific filtering options to check whether a parameter has a value. It is possible to check not only equality, but other mathematical operators, such as: ">", ">=", "<", "<=", "!=". In addition, you can use comparison operators to check whether an element is in an array.
According to the mongodb documentation, the available comparison operators are:
- $eq: Matches values that are equal to a specified value.
- $gt: Matches values that are greater than a specified value.
- $gte: Matches values that are greater than or equal to a specified value.
- $in: Matches any of the values specified in an array.
- $lt: Matches values that are less than a specified value.
- $lte: Matches values that are less than or equal to a specified value.
- $ne: Matches all values that are not equal to a specified value.
- $nin: Matches none of the values specified in an array.
To use these operators, just pass the comparator tag without the "$" symbol. Example:
Request: http://localhost:3000/resources?age=gt:30
Query:
{
"filter": {
"age": {
"$gt": 30
}
}
}
I won't put an example with all operators here, but you can test arithmetic comparison operators on parameters with
values of type string or number, or test the operators of $in
and $nin
on parameters of type array.
Element Filters
Element filters are filters used to deal with parameters that make up the entity's schema. There are two types of element filter possibilities:
$exists: returns elements that have or do not have a specific field
$type: returns elements whose field has a specific type.
Example:
Request: http://localhost:3000/resources?created_at=exists:true&updated_at=exists:false&jobs=type:array
Query:
{
"filter": {
"created_at": {
"$exists": true
},
"updated_at": {
"$exists": false
},
"jobs": {
"$type": "array"
}
}
}
The $exists filter only works with true
or false
values. If a different value is entered, the filter will be
ignored.
The same goes for the $type filter, which only works with valid type values defined in the mongodb documentation ( except deprecated ones):
{
"validTypes": [
"double",
"string",
"object",
"array",
"binData",
"objectId",
"bool",
"date",
"null",
"regex",
"javascript",
"int",
"timestamp",
"long",
"decimal",
"minKey",
"maxKey"
]
}
AND | OR filters
Finally, it is possible to use filters with AND | OR operator. The usage logic follows the arithmetic rule.
To use the AND operator, you must pass the same value twice in a query. Example:
Request: http://localhost:3000/resources?age=gt:30&age=lt:50
Query:
{
"filter": {
"$and": [
{
"age": {
"$gt": 30
}
},
{
"age": {
"$lt": 50
}
}
]
}
}
To use the OR operator, you must enter the values separated by a comma. Example:
Request: http://localhost:3000/resources?age=30,50
Query:
{
"filter": {
"$or": [
{
"age": 30
},
{
"age": 50
}
]
}
}
Populate
If any collection uses references to other objects, in some operations it is interesting to return this information
populated in the object in a single request. For this, the library supports the populate
feature.
There are three ways to add the populate
parameter to the query string:
- Specifying only the field to be populated:
Request: http://localhost:3000/resources?populate=jobs
Query:
{
"populate": {
"path": "jobs"
}
}
- Specifying the field to be populated and which fields should be returned:
Request: http://localhost:3000/resources?populate=jobs;title,salary
Query:
{
"populate": {
"path": "jobs",
"select": {
"title": 1,
"salary": 1
}
}
}
- Specifying the field to be populated, which fields should be returned and a resource filter (useful parameter when the populated field is a list):
Request: http://localhost:3000/resources?populate=jobs;title,salary;salary=gt:3000
Query:
{
"populate": {
"path": "job",
"select": {
"title": 1,
"salary": 1
},
"match": {
"salary": {
"$gt": 3000
}
}
}
}
- Specifying more than one field to be populated:
Request: http://localhost:3000/resources?populate=jobs&populate=currentJob
Query:
{
"populate": [
{
"path": "jobs"
},
{
"path": "currentJob"
}
]
}
There are some rules to consider in populate. The populate must be specified as follows:
populate=field;select;filter
. Soon:
- If you specify only the field to be populated, all field parameters will be returned, and if it is an array, all array elements will be returned;
- If you want to specify which parameters are to be returned from the populated field, you need to specify which fields are to be returned;
- If you want to filter the populated parameters, you need to specify the parameters that should be returned. If you
want to return all object parameters, the
select
parameter must be informed asall
. Example:populate=jobs;all;salary=gt:3000
Rules
- For pagination, you should use
limit
,skip
andpage
only; - For ordination, you should use
sort
only; - For select, you should use
select
only; - For populate, you should use
populate
only; - Anything other than
limit
,skip
,page
,sort
,select
andpopulate
will be considered a filter; - Parameters never contain characters that don't fit the regex
/[^A-z0-9_.]/g
; - Filter values never contain characters that don't fit the regex
/[^\w\[email protected]:]/g
;
Observations
This library is generic. This means that it handles the query based on the query object itself. Therefore, it is not possible to control events such as filter parameters with types incompatible with the types defined in the base. Use proper queries for your API, to prevent implementation errors from being thrown into your app.
Practical Examples
Check out how the configuration of the library in an API works in practice in this project.
Upcoming Features:
- Add a query handling option for Typeorm.
License
Distributed under the Apache License 2.0. See LICENSE
for more information.