fast-sync-memory-dao
v1.0.5
Published
core version of fast sync, synchro service between 2 services distante (backend, mobile/web/service app)
Downloads
6
Maintainers
Readme
FastSync is a powerful and streamlined solutions designed to simplify data synchronization from server side to client application, by consolidating multiple APIs into a single, cohesive interface.
Inspired by Git's approach to version control, FastSync offers a set of functions to provide a robust solution for efficient data synchronization:
- pull()
- push()
- sync()
- hardReset()
FastSync goes beyond traditional synchronization tools by intelligently handling conflicts that may arise due to simultaneous changes on remote ends. Our sophisticated conflict detection mechanism identifies conflicts at the object level, empowering users to provide custom resolution functions.
FastSync supports three distinct conflict resolution strategies to accommodate various use cases:
- LastWriterWins
- TimestampOrdering
- PredefinedRules
Note: when you specify it as predefined rules, you need to provide a custom function to resolve conflicts whenever they are detected.
Supported technologies:
TypeScript server pacakge Dotnet core server pacakge Dart client package
Fast Sync Core, TypeScript
- Server installation
- Server setup
- Server syncable object
- Server syncable object data source
- Server syncalbe repository
- Server Push And Pull
Server installation
"dependencies": {
...
"fast-sync-core": "*.*.*",
"fast-sync-mongoose-dao": "*.*.*",
...
}
Server setup
- In your the entry point of your server, initialize your FastSync package.
- Configure your dao (Data Access Object), When you initialize FastSync package in this exemple we are using mongoose dao.
- Configure the object type to be syncked using setSyncalbeObject method.
- Configure the conflict handler for the configured object type.
async function configureFastSync() {
let fastSync: FastSync = FastSync.getInstance(
new MongooseSyncConfiguration()
);
let conflictsHandler: IConflictsHandler = new ConflictsHandler(
ConflictsResolutionStrategyEnum.PredefinedRules,
conflictsResolutionFunction
);
let repo = new ItemRepository();
await fastSync.setSyncalbeObject(Item.name, repo, conflictsHandler);
}
- Call configureFastSync() in your main func after running the App.
app.listen(port, async () => {
try {
await configureFastSync();
console.log("Configuration completed successfully");
} catch (error) {
console.error("Error configuring Fast Sync:", error);
}
});
Server Syncable Object
Requires fast-sync-mongoose-dao package
you need to extend SyncableObject from fast-sync-core and plug the type metadata schema using plugMetadataSchema
import { SyncableObject } from "fast-sync-core";
import { SyncableSchemaItemBuilder } from "fast-sync-mongoose-dao";
import * as mongoose from "mongoose";
export class Item extends SyncableObject {
public name: string;
public description: string;
public id: string;
constructor() {
super();
}
}
export const ItemSchema = SyncableSchemaItemBuilder.plugMetadataSchema(
new mongoose.Schema({
name: {
type: mongoose.Schema.Types.String,
},
description: {
type: mongoose.Schema.Types.String,
},
})
);
Server Syncable Object Data Source
Now for each SyncableObject you should have a DataSource don't worry every think is setted for you, just create the class.
Note: if you need further method, can just create it there and use it.
import { SyncalbeObjectDataSource } from "fast-sync-mongoose-dao";
import { Item, ItemSchema } from "./item";
export class ItemDataSource extends SyncalbeObjectDataSource<Item> {
constructor() {
super(ItemSchema, Item.name);
}
}
Server Syncalbe Repository
- To interact with the objects, you should only use the repository, you can use the datasource, but like that you break the package.
- Note: note that you don't need to create new method in the repository, because every think is there use the defined method. In some rare extreme case we can create some new methods.
import { SyncalbeRepository } from "fast-sync-core";
import { Item } from "./item";
import { ItemDataSource } from "./item_datasource";
export class ItemRepository extends SyncalbeRepository<Item> {
constructor() {
super(new ItemDataSource(), Item.name);
}
}
Server Push And Pull
- all what you need is to define your 2 apis, one for the /push, the other is the /pull
- we decided to give you aibilty to control your api, and we are taking care of the logic behind.
@Post("/push")
async pushUserObjects(
@Req() req,
@Res() res,
@Body() input: SyncPayload
) {
try{
let syncManager = FastSync.getInstance().getSyncManager()
await this.syncManager.processPush(input)
return this.success(res, {"result":"push was done with success"});
}
catch(err){
return this.error(res,err);
}
}
@Post("/pull")
async pullUserObjects(
@Req() req,
@Res() res,
@Body() metadata: SyncOperationMetadata
) {
try{
let result = await this.syncManager.processPull(metadata)
return this.success(res, result);
}
catch(err){
return this.error(res,err);
}
}
SyncOperationMetadata and SyncPayload and 2 object handled internally in the package, don't worry they will be created from the client part. you don't need to do any thing here.
you can add some middleware for the api if you would like.
Full sync server exemple for mongoose dao
Fast Sync Core, .Net
- .Net server installation
- .Net server setup
- .Net server syncable object
- .Net server syncable object data source
- .Net server syncalbe repository
- .Net server DBContext
- .Net Push And Pull
Net server installation
<PackageReference Include="fast_sync_core" Version="1.0.3" />
<PackageReference Include="fast_sync_EF_dao" Version="1.0.3" />
Net server setup
- In your the entry point of your application, initialize your FastSync package.
- Configure your dao (Data Access Object), When you initialize FastSync package in this exemple we are using Entity Framework dao.
- Configure the object type to be syncked using SetSyncableObject method.
- Configure the conflict handler for the configured object type.
EntityFrameworkSyncConfiguration configuration = new EntityFrameworkSyncConfiguration(dbContextFactory: () => new DataContext(connectionString));
FastSync.GetInstance(syncConfiguration: configuration);
ConflictsHandler conflictsHandler = new ConflictsHandler(resolutionStrategy: ConflictsResolutionStrategyEnum.LastWriterWins);
ItemDataSource dataSource = new ItemDataSource();
ItemRepository repository = new ItemRepository(dataSource: dataSource);
FastSync.SetSyncableObject(typeof(Item), repository: repository,conflictsHandler: conflictsHandler,syncZoneRestriction: SyncZoneRestrictionEnum.Restricted);
Net server syncable object
Requires fast_sync_EF_dao package, for EF configuration
you need to extend SyncableObject from fast_sync_core and plug the type metadata.
using fast_sync_core.implementation.data;
namespace exemple.Item
{
public class Item: SyncableObject
{
public string Name { get; set; } = string.Empty;
public string Description { get; set; } = string.Empty;
}
}
Net server syncable object data source
Now for each SyncableObject you should have a DataSource don't worry every think is setted for you, just create the class.
Note: if you need further method, can just create it there and use it.
using fast_sync_entity_framework_dao.data;
namespace exemple.Item
{
public class ItemDataSource : SyncableObjectDataSource<Item>
{
public ItemDataSource() : base()
{
}
}
}
Net server syncalbe repository
- To interact with the objects, you should only use the repository, you can use the datasource, but like that you break the package.
- Note: note that you don't need to create new method in the repository, because every think is there, Just use the defined methods. In some rare extreme case we can create some new methods in the repository.
using fast_sync_core.abstraction.data;
using fast_sync_core.implementation.data;
namespace exemple.Item
{
public class ItemRepository : SyncableRepository<Item>
{
public ItemRepository(ISyncableDataSource<Item> dataSource) : base(dataSource)
{
}
}
}
Net server DBContext
You will use the same DbContext as usual, Now just extend the FastSyncDataContext instead of DbContext. Don't worry FastSyncDataContext is actually the same DbContext that you know from EF.
All what you need to do is to pass the connection in the constructor.
using exemple.Item;
using Microsoft.EntityFrameworkCore;
namespace fast_sync_entity_framework_dao.data
{
public class DataContext : FastSyncDataContext
{
public DataContext(string connectionString) : base(connectionString)
{
}
public DbSet<Item> Items { get; set; }
}
}
Net Push And Pull
- all what you need is to define your 2 apis, one for the /push, the other is the /pull
- we decided to give you aibilty to control your api, and we are taking care of the logic behind.
[HttpPost("/push")]
[ProducesResponseType(200)]
public async Task<IActionResult> pushAsync([FromBody] SyncPayload syncPayload)
{
try
{
ISyncManager syncManager = FastSync.GetSyncManager();
await syncManager.ProcessPush(syncPayload);
return Ok(new BaseResult(data: "Successfully created", success: true));
}
catch (Exception exception)
{
return Ok(new BaseResult(data: exception, success: false));
}
}
@Post("/pull")
async pullUserObjects(
@Req() req,
@Res() res,
@Body() metadata: SyncOperationMetadata
) {
try{
let result = await this.syncManager.processPull(metadata)
return this.success(res, result);
}
catch(err){
return this.error(res,err);
}
}
SyncOperationMetadata and SyncPayload and 2 object handled internally in the package, don't worry they will be created from the client part. you don't need to do any thing here.
you can add some middleware for the api if you would like.
Full sync server exemple for Entity Framework dao
- Dart client installation
- Client Setup
- Client Syncable Object
- Run the Code Generator
- Client Syncable Object Data Source
- Client Syncalbe Repository
- Http Manager
- Sync Manager
Dart client Installation
dependencies:
# add fast_sync_client to your dependencies
fast_sync_client:
# add fast_sync_hive_dao to your dependencies
fast_sync_hive_dao:
dev_dependencies:
# add the generator to your dev_dependencies
build_runner:
Client Setup
- In your main function of your application, initialize your FastSync package.
- Configure your dao (Data Access Object), When you initialize FastSync package in this exemple we are using Hive dao.
- Configure the object type to be syncked using setSyncalbeObject method, and set the syncZone string for that type specific type.
- initialize the type datasource and repository for that
- Configure the conflict handler for the configured object type.
- Configure the httpManger, using it you will be able to interact with your server, there you will need to implement the push and pull functions.
Note: you can set the syncZone string later, but before the you use the pull and push functions, SyncZone string is unique id, to only sync object that belong to that id.
Exemple: when you specify the user.Id as syncZone, all the object for that type, for that user will be syncked for him.
And when you specify the company.id as syncZone all the object for that type will be syncked for all the users that belong to that company.
you need specify SyncZone, so that the package can know to whom the object will be syncked, if is user.id only the user can see and change these objects.
void main() async {
WidgetsFlutterBinding.ensureInitialized();
await initHive();
FastSync.setSyncConfiguration<HiveSyncConfiguration>(HiveSyncConfiguration());
FastSync.setTypeSyncZone<Item>(SyncZoneRestrictionEnum.restricted,
syncZone: "userId");
FastSync.setHttpManager(HttpManager());
ItemRepository repository = ItemRepository(dataSource: ItemDataSource());
FastSync.setSyncableObject<Item>(
fromJson: Item.fromJson, toJson: Item.intoJson, repository: repository);
runApp(MyApp());
}
Future<void> initHive() async {
var dir = await getApplicationDocumentsDirectory();
Hive.init(dir.path);
Hive.registerAdapter(ItemAdapter());
}
Client Syncable Object
Requires fast_sync_hive_dao package
you need to extend SyncableItemModel from fast_sync_hive_dao and plug the type metadata.
import 'package:fast_sync_hive_dao/fast_sync_hive_dao.dart';
import 'package:hive/hive.dart';
import 'package:json_annotation/json_annotation.dart';
part 'item.g.dart';
@HiveType(typeId: 1, adapterName: "ItemAdapter")
@JsonSerializable(explicitToJson: true)
class Item extends SyncableItemModel {
@HiveField(1)
String name;
@HiveField(2)
String description;
Item({required this.name, required this.description});
@override
factory Item.fromJson(Map<String, dynamic> json) => _$ItemFromJson(json);
@override
Map<String, dynamic> toJson() => _$ItemToJson(this);
static Function get intoJson => _$ItemToJson;
}
Run the Code Generator
Run the generator with flutter packages pub run build_runner build
.
To automatically run it, whenever a file changes, use flutter packages pub run build_runner watch
.
Client Syncable Object Data Source
Now for each SyncableObject you should have a DataSource don't worry every think is setted for you, just create the class.
Note: if you need further method, can just create it there and use it.
import 'package:example/item/item.dart';
import 'package:fast_sync_hive_dao/fast_sync_hive_dao.dart';
class ItemDataSource extends SyncalbeObjectDataSource<Item> {
ItemDataSource();
}
Client Syncalbe Repository
- To interact with the objects, you should only use the repository, you can use the datasource, but like that you break the package.
- Note: note that you don't need to create new method in the repository, because every think is there use the defined method. In some rare extreme case we can create some new methods.
import 'package:example/item/item.dart';
import 'package:fast_sync_client/fast_sync_client.dart';
class ItemRepository extends SyncalbeRepository<Item> {
ItemRepository({required super.dataSource});
}
Http Manager
- HttpManger is abstract class, you need just implement the 2 methods: pull and puss
- Just you need only to make to http calls to your server, don't worry how you create the body of your calls, they will made to you.
import 'dart:async';
import 'dart:convert';
import 'package:dio/dio.dart';
import 'package:fast_sync_client/fast_sync_client.dart';
class HttpManager implements IhttpManager {
final dio = Dio();
HttpManager();
@override
Future<SyncPayload> pull(SyncOperationMetadata metadata) async {
Response response = await dio.post(
'https://www.server.com/pull',
data: json.encode(metadata.toJson()));
if (response.data["success"] == true) {
return SyncPayload.fromJson(response.data["data"]);
}
return throw Exception('pull failed');
}
@override
Future<bool> push(SyncPayload payload) async {
Response response = await dio.post(
'https://www.server.com/push',
data: json.encode(payload.toJson()));
if (response.data["success"] == true) {
return true;
}
return throw Exception('push failed');
}
}
- SyncOperationMetadata and SyncPayload and 2 object handled internally in the package, don't worry they will be created from the client part. you don't need to do any thing here.
Sync Manager
- To Push, Pull, sync, hardReset, Just get syncManager instance using getSyncManager(), and then have fun playing with your data.
import 'package:example/item/item.dart';
import 'package:fast_sync_client/fast_sync_client.dart';
import 'package:flutter/material.dart';
class ItemProvider with ChangeNotifier {
List<Item> items = [];
ISyncableRepository<Item> repository = FastSync.getObjectRepository<Item>();
ISyncManager syncManager = FastSync.getSyncManager();
Future<List<Item>> loadLocalItems() async {
List<Item> localItems = await repository.getAll();
return localItems;
}
Future<void> pull() async {
await syncManager.pull();
}
Future<void> sync() async {
await syncManager.sync();
}
Future<void> push() async {
await syncManager.push();
}
Future<void> hardReset() async {
await syncManager.hardReset();
}
Future<void> resetItemRepo() async {
await repository.hardDelete();
}
Future<Item> saveElement(Item item) async {
item = await repository.add(item);
return item;
}
Future<Item> updateElement(Item item) async {
item = await repository.update(item);
return item;
}
Future<int> getCount() async {
int count = await repository.count();
return count;
}
}
Full sync client exemple for Hive dao
- You can support the library by staring it on Github && liking it on pub or report any bugs you encounter.
- also, if you have a suggestion or think something can be implemented in a better way, open an issue and let's talk about it.