experiments-pkg

Welcome to the experiments-pkg, a library that helps you setting up and running A/B tests within the automotive projects. The library uses the split.io NodeJS SDK under the hood.

Installation

To install the experiments-pkg, run the following command:

npm install @smg-automotive/experiments-pkg @splitsoftware/splitio nanoid

Setting up your first feature flag

1.) Create the feature flag

To create a feature flag, you will visit the split.io app.

1. login to split.io
2. go to Feature flags on the left-hand side and press the button Create feature flag
3. select the traffic type (user/anonymous) and define a speaking name for the feature flag

• user: feature flag targets logged-in users only
• anonymous: feature flag targets all users

4. Set up a dummy configuration for the staging environment, so you can start coding and let someone that is responsible for A/B testing do the rest

• under Treatments, select the treatments you expect to have (usually onand off) and make sure to set a default treatment. The default treatment is what is served for everyone not part of the A/B test.
• under Targeting > Targeting rules, limit the exposure to how many users should be part of the A/B test (e.g. 20% of all the traffic)
• under Targeting > Targeting rules, set the default rule (that is the default what the 20% get)

2.) Make sure the user is remembered

We have 2 active traffic types, user and anonymous. For anonymous feature flags, you need to store a random key as a cookie. As the naming is relevant for GoogleAnalytics, you must name the cookie experiment_user_key (its value is picked up automatically).

A/B testing requires consent and as a result, we can only drop the cookie once consent has been configured. Store the cookie as follows:

// in components/GlobalScriptAndConsentWrapper.tsx
import { nanoid } from 'nanoid';
import { parseCookies } from 'nookies';
import { experimentUserKeyCookieName } from "@smg-automotive/experiments-pkg"

const setExperimentCookie = () => {
    if (typeof window !== 'undefined') {
      // using nookies or anything else you are comfortable with
      const cookies = parseCookies();
      // making sure to keep the existing one
      // use a library like nanoid to generate the user key  
      const userKey = cookies[experimentUserKeyCookieName] ?? nanoid();
      const rootDomain = '.' + window.location.hostname.split('.').slice(-2).join('.');
      setCookie({
        name: experimentUserKeyCookieName,
        path: '/',
        value: userKey,
        domain: rootDomain,
      });
    }
  }

<CookieConsentProvider
  onConsentChanged={(newConsent) => {
    if (hasConsent('experiment', newConsent, 'interaction')) {
      setExperimentCookie();
    }
  }}
  onOneTrustLoaded={(initialConsent) => {
    if (hasConsent('experiment', initialConsent, 'interaction')) {
      setExperimentCookie();
    }
  }}
>
  {children}
</CookieConsentProvider>

3.) Configure the flag

Whenever the connection to split.io fails or in case that we do not have a user key, we need to provide a default treatment. This is configured in code as follows:

import { FeatureFlag } from '@smg-automotive/experiments-pkg';

export const experiments: Record<string, FeatureFlag<string>> = {
  dummyFeature: {
    name: 'dummyFeature',
    defaultTreatment: 'off',
    trafficType: 'anonymous',
  },
} as const;

4.) Create the experiment manager and the getTreatment function

Since we have different brands, we need to instantiate the experiment manager within the application:

import { ExperimentManager, getGetTreatmentFunction } from '@smg-automotive/experiments-pkg';

// this is needed to enable localhost mode
const localhostAuthKey = 'localhost';
const experimentManager = new ExperimentManager({
  autoScoutAuthKey: process.env.AUTOSCOUT24_SERVER_KEY ?? localhostAuthKey,
  motoScoutAuthKey: process.env.MOTOSCOUT24_SERVER_KEY ?? localhostAuthKey,
  debug: "WARN" // optional option -> boolean or LogLevel
});

export const getTreatment = getGetTreatmentFunction(
    experimentManager,
    logException, // method of your choice
);

export default experimentManager;

5.) Query the treatment

On your server-side code, query the treatment for the flags you need to evaluate on the page:

// your own getTreatment function you created in 4.)
import { getTreatment } from '../getTreatment';

// feel free to skip this if you know the traffic type and which key to use
const treatment = await getTreatment(experiments.dummyFeature, {
  query: {}, // used to overwrite the treatment for testing purposes  
  brand: Brand.AutoScout24, // derive brand on the server
  anonymousKey: req.cookies['experiment_user_key'],
  userAccountId: user.accountId,
  attributes: {
      deviceType: "tablet", // pass if you want to target users by device
      language: "de" // pass if you want to target users by language
  }  
});

// do whatever you need to do with the treatment

6.) Tracking

Events are streamed from Google Analytics to split.io. For that reason, we add the experiments running on a page to the page view event. It only passes the data to Google Analytics if the treatment was requested from split.io. When the code fallback has been used (e.g. due to missing consent) or the treatment was overwritten by the query parameter, the data is not passed.

import { getExperimentPageViewData } from '@smg-automotive/experiments-pkg';

// use it where the page view event is triggered. Placement will depend on the project.

const globalTracking = useMemo<GlobalPageViewTracking>(
    () => ({
        event: 'Pageview',
        ...getExperimentPageViewData(experiments),
    }),
    [experiments],
);

localhost mode

If you are developing locally, you likely don't want to talk the split.io. For that, create a file called config/experiment.local.yaml and configure the local treatments

- showroomListing:
    treatment: "on"

More configuration options can be found here.

Development

You can link your local npm package to integrate it with any local project:

cd smg-automotive-example-pkg
npm run build

cd <project directory>
npm link ../smg-automotive-example-pkg

Release a new version

New versions are released on the ci using semantic-release as soon as you merge into master. Please make sure your merge commit message adheres to the corresponding conventions and your branch name does not contain forward slashes /.