Musketeer

A/B testing for Node.js.

// Participate the user in the experiment.
const variant = await Musketeer.participate('MyFirstTest', userID, [
	'A',
	'B',
]);

// Convert the user.
await Musketeer.convert('MyFirstTest', userID);

How does it work?

When you participate a user in an experiment, Musketeer assigns the user to one of the variants. If the user participated the experiment before, the user participates the same variant as before.

Every 8 hours (you can configure the precise timing) Musketeer analyses the conversions of your variants and adjusts the traffic per variant based on it's success. The better a variant is performing, the more traffic it's getting. These adjustments are based on the multi-armed bandit algorithm.

Musketeer#participate(experimentName, userID, variantValues | options)

• experimentName String The name of the experiment.
• userID Any The unique identifier of the user.
• variantValues String[] A list of values per variant. Every variant is given a unique name starting at A, B, and so on.
• options Object Either pass variantValues or options.
• options.type String The type of the experiment. Either bandit, or random.
• options.expiry=28_800 String The expiry of the weights in seconds. This controls how often the weights are recalculated. Defaults to 8 hours. It's generally a good idea to set the expiry longer than the average conversion time of a user. For example, if it takes a user 1.5 days to convert you should set the expiry to at least 1.5 days.
• options.variants Map<String, String> The variants per variant name. { A: 'This is the first variant.', B: 'This is the second variant.' }.

Participates the user in the experiment and returns the value of the variant the user joined. If the user already participated the experiment, the already joined variant's value is returned.

This function returns a Promise and resolves with the variant's value.

Example

const userID = 1;
const variant = await Musketeer.participate('MyFirstTest', userID, {
	type: 'bandit',
	expiry: 60 * 60 * 24,
	variants: {
		A: 'Click me!',
		B: 'Join for free',
	},
});

console.log(variant); // This either prints "Click me!" or "Join for free".

Musketeer#convert(experimentName, userID)

• experimentName String The name of the experiment.
• userID Any The unique identifier of the user.

Counts the user with userID as conversion if not already converted. If you call this function multiple times for the same user, the conversion is only counted once. If the user did not previously join the experiment, the conversion is not counted. This is by design and may change in the future.

This function returns a Promise and resolves with true if the conversions was counted, and false if the user was already converted or did not join the experiment.

Example

const userID = 1;
await Musketeer.convert('MyFirstTest', userID);

Musketeer#info(experimentName)

• experimentName String The name of the experiment.

What's next

• Automatically declare a winning variant after a configurable