ucb
v3.2.0
Published
An upper confidence bounds multi-armed bandit algorithm
Downloads
12
Maintainers
Readme
ucb
An upper confidence bounds algorithm for multi-armed bandit problems
This implementation is based on Bandit Algorithms for Website Optimization and related empirical research in "Algorithms for the multi-armed bandit problem". In addition, this module conforms to the BanditLab/2.0 specification.
Get started
Prerequisites
- Node.js 6.x+ (LTS track)
- npm
Installing
Install with npm
(or yarn
):
npm install ucb --save
Caveat emptor
This implementation often encounters extended floating point numbers. Arm selection is therefore subject to JavaScript's floating point precision limitations. For general information about floating point issues see the floating point guide.
Usage
Create an optimizer with
3
arms:const Algorithm = require('ucb'); const algorithm = new Algorithm({ arms: 3 });
Select an arm (exploits or explores, determined by the algorithm):
algorithm.select().then((arm) => { // do something based on the chosen arm });
Report the reward earned from a chosen arm:
algorithm.reward(arm, value);
API
Algorithm(config)
Creates a new optimization algorithm.
Arguments
config
(Object
): algorithm instance parameters
The config
object supports two optional parameters:
arms
(Number
, Integer): The number of arms over which the optimization will operate; defaults to2
Alternatively, the state
object resolved from Algorithm#serialize
can be passed as config
.
Returns
An instance of the ucb optimization algorithm.
Example
const Algorithm = require('ucb');
const algorithm = new Algorithm();
assert.equal(algorithm.arms, 2);
Or, with a passed config
:
const Algorithm = require('ucb');
const algorithm = new Algorithm({ arms: 4 });
assert.equal(algorithm.arms, 4);
Algorithm#select()
Choose an arm to play, according to the optimization algorithm.
Arguments
None
Returns
A Promise
that resolves to a Number
corresponding to the associated arm index.
Example
const Algorithm = require('ucb');
const algorithm = new Algorithm();
algorithm.select().then(arm => console.log(arm));
Algorithm#reward(arm, reward)
Inform the algorithm about the payoff from a given arm.
Arguments
arm
(Number
, Integer): the arm index (provided fromAlgorithm#select()
)reward
(Number
): the observed reward value (which can be 0 to indicate no reward)
Returns
A Promise
that resolves to an updated instance of the algorithm. (The original instance is mutated as well.)
Example
const Algorithm = require('ucb');
const algorithm = new Algorithm();
algorithm.reward(0, 1).then(updatedAlgorithm => console.log(updatedAlgorithm));
Algorithm#serialize()
Obtain a plain object representing the internal state of the algorithm.
Arguments
None
Returns
A Promise
that resolves to a stringify-able Object
with parameters needed to reconstruct algorithm state.
Example
const Algorithm = require('ucb');
const algorithm = new Algorithm();
algorithm.serialize().then(state => console.log(state));
Development
Contribute
PRs are welcome! For bugs, please include a failing test which passes when your PR is applied. Travis CI provides on-demand testing for commits and pull requests.
Workflow
- Feature development and bug fixing should occur on a non-master branch.
- Changes should be submitted to master via a Pull Request.
- Pull Requests should be merged via a merge commit. Local "in-process" commits may be squashed prior to pushing to the remote feature branch.
To enable a git hook that runs npm test
prior to pushing, cd
into the local repo and run:
touch .git/hooks/pre-push
chmod +x .git/hooks/pre-push
echo "npm test" > .git/hooks/pre-push
Tests
To run the unit test suite:
npm test
Or, to run the test suite and view test coverage:
npm run coverage