angular-ab-tests
v1.3.1
Published
Easy and descriptive way to setup complex AB tests and multivariate tests in Angular2+
Downloads
847
Maintainers
Readme
AngularAbTests is an angular module that helps you setting up easily and clearly any AB or multivariate test.
It will make your tests easy to debug and understand, regardless of how complex they are, of how many versions you are setting up, or even of how many concurrent tests you are running.
Update: version 1.3.0 has been released, here is the changelog
- Added support for server side rendering.
Contents
- List of features
- Quick introduction to usage
- Why this plugin is good for you
- How to set up a quick demo to play around
- Full documentation part 1: Initializing
- Full documentation part 2: Usage
- Full documentation part 3: Tips
- Server side rendering
Features
- Set up an A/B test with a simple and descriptive code (the module makes available just one simple structural directive)
- A/B tests are valid across different pages, using cookies with the expiration you want
- Test multiple versions with changes as complicated as you need, while still keeping the code clean and understandable
- Effortlessly set up any number of independent A/B tests, being sure that they won't clash in the code
- Select a special version for crawlers, so you won't affect your SEO while you are running the test
Usage in short
Add it to the list of dependencies using npm: https://www.npmjs.com/package/angular-ab-tests
npm install angular-ab-tests --save
Set up an A/B test including the module with forRoot
method:
@NgModule({
imports: [
AbTestsModule.forRoot([
{
versions: [ 'old', 'new' ],
versionForCrawlers: 'old',
expiration: 45,
},
]),
],
exports: [
AbTestsModule,
],
})
Wrap fragments of HTML inside the structural directive named abTestVersion
, marking them to belong to one or more of the versions of your A/B test.
<ng-container *abTestVersion="'old'">
Old version goes here
</ng-container>
<ng-container *abTestVersion="'new'">
New version goes here
</ng-container>
Why to use this plugin
- You can create several different versions, as complex and as big as you need, without filling your HTML with unnecessary code. This will make your A/B test less error prone, and also it will make it easier to remove the loser versions after the test, because the code is clear and descriptive.
- Versions that are not selected are automatically removed from change detection at initialization, so no performance issues.
- You can easily span your tests across different pages reading the same cookie, with no additional code.
- You can maintain as many different A/B tests you want without risking them to clash in the code.
What about simple A/B tests? Why should I use AngularAbTests for a simple test as well?
Usually, to set up a small A/B test (changing a color, removing or adding a div, etc) people use client side tools like Google Optimize.
This approach can potentially affect user experience, because Google Optimize has to change parts of the page depending on the specific version selected, and, if this happens while the user is already looking at the page, we have the effect called "page flickering". To prevent page flickering Google Optimize introduced a "hide-page tag", i.e. a script that hides the page until the external call to Google Optimize server has responded.
Now, usually Google Optimize tag loads fast, but you cannot always rely on external calls, especially in conditions of low network; in the worst scenario, if Google Optimize server doesn't respond, the hide-page tag gets unblocked after the threshold of 4 seconds.
This means that, even if your server has responded in 150 milliseconds, your user won't be able to see anything in the page until the 4 seconds have expired.
Are you sure you want to risk this? With AngularAbTests you can set up a simple A/B test easily and cleanly directly in the code, this means that you can get rid of the hide-page tag, and let Google Optimize focus only on data collection.
Set up a demo
You can setup a simple demo to play around with the plugin and see how it works.
- Execute
git clone [email protected]:adrdilauro/angular-ab-tests.git
- Navigate to repository folder
- Execute
npm install
- Execute
ng serve
- Visit
http://0.0.0.0:4200
in your browser
The demo contains a simple A/B test serving two different components depending on the chosen version. You can play around, add more tests / versions, and explore all the configuration options.
Keep in mind that in the demo angular-ab-tests
is defined as a local module and not via npm, so if you want the demo code to work for your app you have to replace the imports:
import { AbTestsModule } from './modules/angular-ab-tests/module'; // WRONG! Works only in the demo
import { AbTestsModule } from 'angular-ab-tests'; // CORRECT, works when loading the module from npm
AngularAbTests is fully covered by specs: spec file is located in src/app/directive.spec.ts, to run it in the demo you have to navigate in the root folder and execute ng test
Documentation 1: Initializing
AngularAbTests declares just one directive called *abTestVersion
. The directive is structural and it's used to wrap parts of the HTML that you want to A/B test.
In order to use this directive, you need to import the module AbTestsModule
wherever you need it.
Besides importing the module, the other thing you need to do is initialize the global service the directive reads from: this service is used to store all the running A/B tests, and when it's called by the directive it returns the appropriate test with the version that is currently chosen.
The service needs to be global, so it needs to be initialized at root level: for this reason, AngularAbTests provides a forRoot
method to be used at the root of your application:
@NgModule({
imports: [
AbTestsModule.forRoot([
{
// Configuration for the first test
},
{
// Configuration for the second test
},
// etc
]),
],
})
export class AppModule {}
I'll soon explain in detail (see Documentation 2: Usage) both the usage of the directive and the options available in each configuration object. Before coming to that, I want to give some practical tips about the set up process.
Best practice for setting up AngularAbTests
1 - Set up the root import
The best way to set up AngularAbTests is to include the forRoot
call in your CoreModule
, this is called the "forRoot pattern" (see Angular's official documentation for details: in a nutshell, when you need a service to be global under all circumstances, you cannot risk it to be included in modules that are lazy loaded, otherwise the lazy loaded module will be provided of a copy of the original service).
So, if you follow the Angular best practice and have your own CoreModule
, that's the best place to configure AngularAbTests:
@NgModule({
imports: [
SomeModuleWithProviders,
AnotherModuleWithProviders,
AbTestsModule.forRoot([
{
// Configuration for the first test
},
{
// Configuration for the second test
},
// etc
]),
],
})
export class CoreModule {
// Content of the CoreModule, usually a guard against double loading
}
If you are setting up a lot of tests, you might want to clean up your CoreModule
and extract the AngularAbTests logic into a separate one. Create a separate file called tests.module.ts
:
import { NgModule } from '@angular/core';
import { AbTestsModule, AbTestOptions } from 'angular-ab-tests';
export const abTestsOptions: AbTestOptions[] = [
{
// Configuration for the first test
},
{
// Configuration for the second test
},
// etc
];
@NgModule({
imports: [
AbTestsModule.forRoot(abTestsOptions),
],
})
export class TestsModule {}
In order to clean up better your module, you can declare the configuration options separately as a constant of type AbTestOptions[]
: type AbTestOptions
is imported from angular-ab-tests
. Again, for a detailed description of configuration options, see the second part of the documentation.
To complete your refactoring, you then import your TestsModule
into CoreModule
:
@NgModule({
imports: [
SomeModuleWithProviders,
AnotherModuleWithProviders,
TestsModule,
],
})
export class CoreModule {
// Content of the CoreModule, usually a guard against double loading
}
2 - Set up the directive
The best place to set up the directive *abTestVersion
is the SharedModule
, in order not to accidentallly forget to import it in every single module. Shared modules are a recommended way to organize your shared dependencies.
Simply configure your SharedModule
to import and re-export the bare AbTestsModule
, like this:
@NgModule({
imports: [ AbTestsModule ],
exports: [ AbTestsModule ],
})
export class SharedModule {}
To see quickly this whole configuration in action, please set up the demo.
To read more about SharedModule
and CoreModule
, you might find useful this list of module patterns from the official docs.
Documentation 2: Usage
The config interface
This is how the configuration AbTestOptions
is defined:
export interface AbTestOptions {
versions: string[];
scope?: string;
versionForCrawlers?: string;
expiration?: number;
domain?: string;
weights?: {
[x: string]: number,
};
}
When you setup the module using forRoot
, you have to pass as argument an array of objects that respect this interface. Let's go over all the options:
versions
: all the versions that your test is going to use (an array of strings): in order not to get confused, better using alphanumeric trimmed strings (anyway, if you accidentally mistype a version later AngularAbTests will raise an exception).scope
: if you are setting up more than one test at the same time, you have to specify a scope to distinguish them; if left undefined it will be automatically the string'default'
. AngularAbTests will raise an exception if the same scope is used twice.versionForCrawlers
: use this field if you want one of the versions to systematically be shown to crawlers (you don't need to care about this if SEO is not important for you); of course the version needs to be one of the declared ones.expiration
: the number of days you want the cookie to persist; if left undefined the cookie will expire when the browser session ends.domain
: domain for the cookie (if left undefined it will use the standard domain).weights
: a hash of integers< 100
, associated to the versions you have defined: use this option if you want some of your versions to appear mor frequently than the others. Weights for versions you didn't specify will be equally distributed.
Examples of weight configurations for a list of versions ['v1', 'v2', 'v3']
:
{ v1: 40 }
will produce a40%
chance to extractv1
, and30%
for both the other two versions{ v1: 90, v2: 9 }
will produce a90%
chance to extractv1
, a9%
to extractv2
, and a remaining1%
to extractv3
{ v1: 50, v2: 55 }
will raise an exception because50 + 55 = 105 > 100
{ v1: 40, v2: 30, v3: 35 }
will raise an exception because40 + 30 + 35 = 105 > 100
{ v1: 40, v6: 45 }
will raise an exception because versionv6
hasn't been declared
Example of a correct complete configuration
AbTestsModule.forRoot(
[
{
versions: [ 'v1', 'v2', 'v3' ],
scope: 'versions',
expiration: 45,
weights: { v1: 45, v3: 100/3 }
},
{
versions: [ 'red', 'green', 'blue' ],
scope: 'colors',
versionForCrawlers: 'green',
},
{
versions: [ 'old', 'new' ],
domain: 'xxx.xxx',
versionForCrawlers: 'old',
weights: { old: 60 }
},
]
)
The directive
The directive abTestVersion
wraps a portion of HTML and decides whether showing it or not depending on the following factors:
- Does the version stored in the cookie match the one declared in the directive?
- If the call comes from an SEO crawler, has this version been chosen to be shown to crawlers?
This is the most basic implementation of the directive:
<ng-container *abTestVersion="'v1';scope:'default'">
<!-- Content -->
</ng-container>
The "scope" is necessary to map to the correct test if you set up more than one: if you are pointing to the default test, you can omit the scope, like this:
<ng-container *abTestVersion="'v1'">
<!-- Content -->
</ng-container>
You can associate one block of HTML to two versions: instead of writing the directive twice,
<ng-container *abTestVersion="'v1'">
<!-- Content -->
</ng-container>
<ng-container *abTestVersion="'v2'">
<!-- Very same content -->
</ng-container>
you can simply declare the abTestVersion
directive once, separating versions with a comma:
<ng-container *abTestVersion="'v1,v2'">
<!-- Content -->
</ng-container>
Versions should be separated by comma without spaces; however don't worry too much about mistyping, because if you accidentally add a space AngularAbTests will realise one of your versions doesn't match any of the declared ones, and raise an exception.
As I have already said, in the configuration you can specify that a version is always going to be shown to SEO crawlers; you can do this at directive level as well, forcing a specific block to be shown to crawlers, regardless of how you had set that test up:
<ng-container *abTestVersion="'v1';forCrawlers:true">
<!-- Content -->
</ng-container>
Even if version v1
wasn't configured to be shown to SEO crawlers, this specific block will be shown to crawlers.
Remember one thing: if you don't specify any versionForCrawlers
in your configuration, nor add manually forCrawlers
in any of your directives, this automatically implies that none of the versions will be rendered when a SEO crawler visits your page; but of course, if your website is an application accessible only via login, that doesn't need to worry about SEO, this would be perfectly fine.
Manually read / set a specific version during runtime
First, you need to inject the token AbTestsService
:
import { Component } from '@angular/core';
import { AbTestsService } from 'angular-ab-tests';
@Component({
selector: '...',
templateUrl: '...',
})
export class MyComponent {
constructor(private abTestsService: AbTestsService) {
//...
}
}
You can then call the public methods getVersion
and setVersion
, specifying, as usual, the test scope, if you want to work on a specific test that is not the default one.
// This retrieves the version associated to default scope
this.abTestsService.getVersion();
// This retrieves the version associated to a scope different by the default one,
// in case you are running multiple tests
this.abTestsService.getVersion('my-scope');
// This sets the version associated to default scope
this.abTestsService.setVersion('xxx'); // It raises an exception if `xxx` is not whitelisted
// This retrieves the version associated to a scope different by the default one,
// in case you are running multiple tests
this.abTestsService.setVersion('xxx', 'my-scope');
IMPORTANT: when you use setVersion
, the version only changes for pieces of HTML that have not been rendered yet.
This behaviour is logical: for whatever reason you are changing manually the version of a test, you don't want the change to apply to parts of the page that are already rendered, because the user would see a weird flickering.
So, if you want to force a version, please ensure that you do it BEFORE rendering any HTML block affected by that test: otherwise, the user might end up seeing, in the same page, HTML blocks corresponding to different versions.
Debugging cookies
I strongly suggest you to use the Chrome extension called EditThisCookie.
Documentation 3: Tips
1 - Apply the directive on a ngContainer
The HTML tag <ng-container>
is an empty tag that is not rendered, it's used only to wrap a portion of HTML inside a structural directive. I suggest using this tag to associate a portion of HTML to an AB test version:
<!-- Recommended implementation -->
<ng-container *abTestVersion="'old'">
<component-for-version-old>
</component-for-version-old>
</ng-container>
<!-- Not recommended because you render an unnecessary div -->
<div *abTestVersion="'old'">
<component-for-version-old>
</component-for-version-old>
</div>
<!-- Not recommended because you are mixing the logics of your AB test with the logic of your app -->
<component-for-version-old *abTestVersion="'old'">
</component-for-version-old>
2 - Pass only static values to the directive
As already mentioned, change detection is disabled for anything contained in a version that is not rendered, so you never risk a performance issue. However, change detection is still enabled on the wrapping directive, so in order to reduce computation to the minimum possible you should pass only static values to the directive:
<!-- Recommended implementation -->
<div *abTestVersion="'old'">
<!-- Content -->
</div>
<!-- Not recommended because "getOldVersion()" will be fired at every change detection tick -->
<div *abTestVersion="getOldVersion()">
<!-- Content -->
</div>
3 - Better to avoid nested directives
You should keep your code logic clean and easy to debug: be careful not to nest calls to directives associated to the same test, otherwise you might get unpredictable behaviours:
<!-- Not recommended -->
<ng-container *abTestVersion="'old'">
<ng-container *abTestVersion="'new'">
</ng-container>
</ng-container>
In theory, if you nest directives associated to different tests, you are not doing anything wrong; however, there is a high chance that if you are doing that the statistical results of your tests will clash.
<!-- In principle this is not wrong, but the results of your tests might not come out consistent -->
<ng-container *abTestVersion="'old';scope:'firsttest'">
<ng-container *abTestVersion="'new';scope:'secondtest'">
</ng-container>
</ng-container>
So, better not to nest two directives of type abTestVersion
.
How to be sure that you are not nesting two directives? Unfortunately with the decomposition of an HTML page into Angular components there is no definitive way of ensuring this, you'll have to organize your code smartly.
4 - Ensure your tests are statistically consistent
Be careful not to produce a false positive by running two AB tests in the same time: there are many blogs covering this topic, for instance one I found is this.
Server Side Rendering
There are a couple of ways to make this module work in SSR (thanks @alirezamirian for this addition):
1 - Using AbTestsServerModule
The module AbTestsServerModule
is an extension that overrides the services used to handle userAgent and cookies: it replaces browser entities such as document
and window
with SSR-friendly services that do the same thing.
You need to import it in your app.server.module.ts
, in addition to the other imports discussed in the previous points.
import { NgModule } from '@angular/core';
import { AbTestsServerModule } from 'angular-ab-tests';
@NgModule({
imports:[
// ...
AbTestsServerModule
]
})
export class AppServerModule {}
AbTestServerModule
optionally depends on REQUEST
from @nguniversal/express-engine and CookiesService
from @ngx-utils/cookies for detecting crawlers and manipulating cookies.
Note that, even if they are not provided, both modules should be installed.
2 - Providing necessary services
Alternatively, you can also provide CookieHandler
and CrawlerDetector
services yourself in your server module:
import { NgModule } from '@angular/core';
import { CrawlerDetector, CookieHandler } from 'angular-ab-tests';
import { MyOwnCrawlerDetector, MyOwnCookieHandler } from '...';
@NgModule({
providers: [
// ...
{
provide: CrawlerDetector,
useClass: MyOwnCrawlerDetector,
},
{
provide: CookieHandler,
useClass: MyOwnCookieHandler,
},
],
})
export class AppServerModule {}
The class MyOwnCrawlerDetector
must expose a method isCrawler(): boolean
, that, as the name suggests, returns true if the page is being visited by a crawler.
The class MyOwnCookieHandler
must expose methods get
and set
, with the following interfaces:
get(name: string): string
set(name: string, value: string, domain?: string, expires?: number): void
Obviously, in your own implementation of MyOwnCookieHandler
, you don't have to worry about where the parameters name
, domain
, etc come from: you just need to add them to the cookie, if they are present, with whatever cookie-writing interface you are using.