aset
v0.1.0
Published
Scalable Redis Sorted Sets
Downloads
2
Readme
ASet - Scalable Redis Sorted Sets
Redis Sorted Sets that scales.
WARNING! Highly experimental. Use at your own risk!
INTRO
To see what problem ASet is tackling, please read ASet — scaling sorted sets
REQUIREMENTS
- Node.js v6+
- Redis v4+
INSTALLATION
npm install aset
USAGE
const ASet = require("aset")
const setKey = "foo"
// Minimum recommended config (4 servers)
const clientOptions = {
c1: {
ared: {
s1: {
host: "127.0.0.1",
port: 8124,
redis: {
r1: {
host: "127.0.0.1",
port: 6379
}
}
},
s2: {
host: "127.0.0.1",
port: 8125,
redis: {
r2: {
host: "127.0.0.1",
port: 6380
}
}
}
},
replication: 2,
writePolicy: 2
},
c2: {
ared: {
s3: {
host: "127.0.0.1",
port: 8126,
redis: {
r1: {
host: "127.0.0.1",
port: 6381
}
}
},
s4: {
host: "127.0.0.1",
port: 8127,
redis: {
r2: {
host: "127.0.0.1",
port: 6382
}
}
}
},
replication: 2,
writePolicy: 2
}
}
const aset = new ASet(clientOptions, setKey, () => {
const score = 101
const member = "bar"
aset.add(score, member, (error, result) => {
if (error) {
throw error
}
console.log(result) // OK
})
// OR
aset.get((error, result) => {
if (error) {
throw error
}
console.log(result) // [score, member]
})
})
SCALING
c1, c2, c*
are into how many pieces the sorted set will be split. Add more to scale write performance.s1, s2, s*
are servers used for replication. Add more to scale read performance.r1, r2, r*
are just Redis instances. Add as many per server as cores are available.- It is important that all
c, s and r
are unique (due to ARed requirements). - Currently if you change
s and r
, you need to use ARed rebalancer. But if you changec
you will need to rebalance data manually.
BEST PRACTICES
- It's better to separate ADD and GET. This way, in case you only have 1 GET process, it will perform at it's max.
- You can scale ADD by adding more of the processes. I recommend using PM2 and a reverse proxy with NGINX for that.
- You can only scale GET if order of the results is not important or you have your own mechanism to ensure the order.
TESTING
NOTE: Requires Redis (localhost and ports 6379-6832) to be installed (4 instances for full testing)
npm test
LINTING
npm run lint
CONTRIBUTING
Go nuts! Just don't forget to test and lint. Credit will be given where it's due.
FUTURE
- Create rebalancer.
- Benchmarks
- Lots of fixes and optimizations