PBS KIDS Lighthouse CLI (PBSKLCLI)

Description

This package is designed to make it easy to run Lighthouse against all pages on your Next.js website. It is especially targeted at a CI context, such as Github Actions.

Note: It is possible that we will support other frameworks in the future, but for now we're only going to support Next.js.

Installation

From your Next.js website, install it as a dependency:

npm i -D pbs-kids-lighthouse-cli

Configuration

Ignore Files

You will want to add /.lighthouseci and pbsk-lighthouse-configs/.envs.json to your .gitignore file. This is where all reports are output by default.

Also it's recommended to have you editor ignore those files. For VSCode, add this to your .vscode/settings.json file:

{
  "search.exclude": {
    ".lighthouseci/**": true,
  }
}

Config Files

The configs defined in this repo, in src/configs, are used by default. You can override them by copying those into folder called pbsk-lighthouse-configs to the root of your project:

mkdir pbsk-lighthouse-configs
cp node_modules/pbs-kids-lighthouse-cli/src/configs/* pbsk-lighthouse-configs/

There are two types of configs: the General Config and Profile Configs.

General Config

general-config is the main config. It can be a .js or .json file but .js is recommended. This config defines an object that contains the following values:

• serverTech [string] - The server technology used by your website. Possible values are:

  • nextjs - used in the context of Next.js website projects. With minimal configuration, PBSKLCLI will automatically generate the list of routes to test.
  • unknown - used for websites that run on any server tech, but will require a list of static routes. See sample config file @ src/configs/general-config-unknown.js. You can see it in action by running npm ci && npm run lighthouse --customgeneralconfig=general-config-unknown directly from this package.

• onlyCategories [Array<string>] - Optional - An array of strings that represent the Lighthouse categories to test against. Possible values are: performance, accessibility, best-practices, seo, pwa. Defaults to all categories.

• baseUrl [string] - The base URL of the website to test against. For Next.js projects, this value will default to localhost. It can be overridden with the --env runtime flag.

• skipRoutes [Array<string>] - An array of regex patterns used to match routes we want to skip.

• profiles [Array<Object>] - An object that defines the profiles to run Lighthouse against. Here is the expected format:

      profiles: [
        // The default profile that ships with PBSKLCLI. You can replace it with your own:
        {
          'name': 'pbs-kids-standard',
          'default': true,
        },
        {
          'name': 'pbs-kids-video',
          'matchRoutes': 'video',
        },
      ],

  Each route will be matched against each non-default profile's matchRoutes pattern, in the order they are listed in this config. Routes will later be tested against either their first matching profile, or against the default profile. See the section below for more information about the Profile Configs themselves.

Specific to Next.js

• serverPortOverride [number] - The local website server that PBSKLCLI spins up. Defaults to port 9876.

• dynamicRoutes [Object] - An object that defines dynamic routes and the params they need to be passed. See example below.

    dynamicRoutes: {
      '/some/dynamic/route/[exampleParamName]': [
        {
          'exampleParamName': 'example-value',
        },
      ],
      '/videos/playlist/[...slugsIds]': [
        {
          'slugsIds': 'creature-powers/1243436',
        },
      ],
    },

  This example config would add /some/dynamic/route/example-value and /videos/playlist/creature-powers/1243436 to the list of routes tested.

• queryStringRoutes [Object] - An object that supports passing query string parameters to specific routes. See example below.

    queryStringRoutes: {
      'some-route/to-match$': [
        {}, // Output the route without params. Omit this array item if you don't want to run Lighthouse against the route without params.
        {
          'someParamName': 'someParamValue',
        }
      ],
    },

  This example config would add /some-route/to-match and some-route/to-match?someParamName=someParamValue to the list of routes tested.

Specific to unknown server tech

• staticRoutes [Array<string>] - An array of static routes to test, relative to the website's baseUrl. This is required for websites that run on any server tech. See sample config file @ src/configs/general-config-unknown.js.

Profile Configs

See the default profiles that ship with PBSKLCLI in src/configs/profiles. You can copy from or use these profiles as-is.

The values within the Profile Configs end up dynamically populating the ci.assert and ci.collect.settings.budgets portions of the lighthouserc.json Lighthouse configuration file that ultimately gets passed into the Lighthouse tool for each profile.

For documentation of the values that can be used in the Profile Configs, see the Lighthouse docs:

• Assertions
• Budgets

LHCI Server Configuration

In order to submit your Lighthouse results to an LHCI server, there are two required configuration values and two optional ones. The required values are the server base URL and the upload token. The optional values are the basic auth username and password which you only need if your server is behind basic authentication. These values can be set as environment variables or in a shell script like this:

#!/usr/bin/env bash

export PBSKLCLI_LHCI_SERVER_BASE_URL="https://lhciserver.yourwebsite.com/"
export PBSKLCLI_LHCI_UPLOAD_TOKEN="token-goes-here"

export PBSKLCLI_LHCI_BASIC_AUTH_USERNAME="username"
export PBSKLCLI_LHCI_BASIC_AUTH_PASSWORD="password"

If you opt for the shell script, you will want to use the envsource runtime flag to point to the file, like so:

npm run lighthouse --outputtarget=lhci --envsource="./.env.lhci"

Usage

Run Lighthouse Against Your Website

The most direct way, after installing this package in your Next.js project, is:

npx pbsklighthouse [route patterns as arguments] [runtime flags, eg. --name=value] [shell flags]

However, we recommend you add these scripts to your package.json file:

    "lighthouse": "pbsklighthouse",
    "lighthouse:standard": "npm run lighthouse --preset=lighthouse:recommended",
    "lighthouse:strict": "npm run lighthouse --preset=lighthouse:all",

Given that the first script (lighthouse) is present, you can run Lighthouse against your website with the following command syntax:

# Don't forget to build your project first!
npm run build

npm run lighthouse [route patterns as arguments] [runtime flags] -- [shell flags]

This is the method that the rest of this document will utilize in example commands.

Route Patterns as Arguments

There are a few ways to run Lighthouse against specific routes. See the following examples:

# Run against all routes found:
npm run lighthouse

# Run against just the homepage:
npm run lighthouse /

# Run against routes that end with word `video$`:
npm run lighthouse 'video$'

# Run against the homepage and any route that ends with `1243436`:
npm run lighthouse / '1243436$'

Runtime Flags

The following flags can be passed to the CLI, in the format --name=value.

| Flag Name | Default | Possible Values | Description | |---|---|---|---| | limit | -1 | Integers | Specify a max number of URLs per profile to run tests against. | | offset | -1 | Integers | Specify the position in the URL list to start at. URLs before that position will be skipped. | | envsource | null | File path | Specify the path to a shell script that exports environment variables. | | port | 9876 | Any available port number | Override the port that the Next.js server is run on. | | runcount | 1 | Integers | This is the number of times to run Lighthouse against each page. This is useful to get a more accurate score, as the first run is often slower than subsequent runs. | | batchsize | -1 | Integers | The number of URLs to test in each batch. If not set, all URLs will be tested in one batch. This is useful for large sites, since if a backend error occurs, the entire batch will fail, but the rest of the URLs will still be tested. | | startatbatch | 1 | Integers | Specifies which batch number to start at. Useful if a failure occurs, so you can re-run the failed batch instead of many batches. | | cachewarming | false | true, false | If set to true, before running Lighthouse against all pages in your website, a CURL command will be run to warm any backend-cached data. This is useful to get a more accurate score, as the first run is often slower than subsequent runs if the cache is not warmed. For this to work, see this. | | outputtarget | filesystem | filesystem, temporary-public-storage, lhci | If set to temporary-public-storage, the Lighthouse reports will be uploaded to a temporary public storage bucket and the URLs will be printed to the console. This is useful for running Lighthouse against a project in a CI environment, where you want to be able to view the reports after the build is complete. lhci allows PBSKLCLI to submit your Lighthouse results to an LHCI server. See the section LHCI Server Configuration for more details. | | outputtargetdir | ./.lighthouseci | Directory path | Applied only if outputtarget is set to filesystem. Manually sets the directory where the Lighthouse reports will be written. | | lhcidescription | `` | String | Applied only if outputtarget is set to lhci. A description that will be included in the run title when submitted to the LHCI server. | | createlisting | false | true, false | If set to true and outputtarget is set to filesystem, an index.html file will be created in the output target directory. This file will contain a listing of all runs of Lighthouse found in the output target directory. | | preset | null | Lighthouse builtin presets | Force Lighthouse to use one of its builtin presets. | | profile | null | Profile names that are defined in general-config | Only can be used in conjunction with the shell flag --list-routes. | | env | local | Names defined in pbsk-lighthouse-configs/.envs.json or a base URL (see below) | Essentially tells PBSKLCLI what the base URL is to the website it will test against. | | onlycategories | Comma-delimited combination of strings: performance, accessibility, best-practices, seo, pwa | Limit the categories that Lighthouse will run against. | If set, only the specified categories will be tested. Documentation here: https://github.com/GoogleChrome/lighthouse/blob/main/docs/configuration.md#settings-objectundefined | | customgeneralconfig | null | Filename (without extension) of a custom general-config, as a string. | If set, a custom general-config will attempt to load. | | urlcheckonly | false | true, false | If true, the script will only perform a simple fetch() of each URL to check its status code, and not run the Lighthouse tests. | | debug | false | true, false | If true, increases logging verbosity. |

Example usage:

npm run lighthouse --port=234 --debug=true --outputtarget=temporary-public-storage --limit=1

Shell Flags

• --list-routes - Testing will be skipped in favor of outputting a list of routes Lighthouse would test against, given the current configuration. Make sure you have a current build of your project before using this flag. Can be used in conjunction with --profile runtime flag. Also accepts a filepath to output the list of routes to, in the form of --list-routes=/path/to/output.txt.
• --suppress-failing-exit-code - Always return a successful exit code, even if Lighthouse reports errors.
• --suppress-nonerror-output - Non-error output will be suppressed for more scannable output.

Example usage:

npm run lighthouse --port=999 -- \
  --suppress-failing-exit-code \
  --suppress-nonerror-output

Running Against Multiple Environments

You can also create file pbsk-lighthouse-configs/.envs.json (remember to git-ignore this file to avoid leaking secrets!) in your project which can be used to specify the URLs that this tool can run Lighthouse against. The file's contents should look like this:

{
  "prod": "https://some-website.org",
  "stage": "https://some-website-stage.pbskids.org",
  "dev": "https://some-website-dev.pbskids.org",
  "otherenvironmentname": "https://other-environment-name.pbskids.org",
}

This will allow you to run Lighthouse against prod (or the other environments), (assuming the remote routes match your local project) by running:

npm run lighthouse --env=prod

Alternatively, you can use the --env flag to specify the base URL of environment you want to run Lighthouse against.

npm run lighthouse --env=https://some-website.pbskids.org

Cache Warming

Sometimes the first load of a page can be slow on the backend and cause Lighthouse to either timeout or give your pages poorer scores than it should. We can mitigate this by warming the website's internal cache -- by simply requesting each page on the website before running Lighthouse against them.

To support cache warming, you should add this to your next.config.js

(async () => {
  // neat trick to run something after server starts.
  // source: https://github.com/vercel/next.js/discussions/15341#discussioncomment-5452709
  if (process.env.__WARM_CACHE_AFTER_START) {
    const { warmLocalCache } = await import('pbs-kids-lighthouse-cli/nextjs/server.js');
    warmLocalCache();
  }
})();

...and this to your package.json:

    "startandwarmcache": "__WARM_CACHE_AFTER_START=true next start",

Once you have completed these steps, you will be able to run Lighthouse with cache warming enabled like this:

npm run lighthouse --cachewarming=true

Use in a Github Action

cd /path/to/myproject
mkdir -p .github/workflows
cp node_modules/pbs-kids-lighthouse-cli/sample-ci.yml .github/workflows/lighthouse.yml

See Github documentation for further details about workflows: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions.

Development

Tests

They will be automatically run on PR or push, but you should also run them locally:

npm test all # or without the "all" to see the other options

Other Useful Documentation

• What GA is doing with Lighthouse: https://github.com/pbs/prism
• For PR checks - see: https://web.dev/lighthouse-ci/#github-status-checks
• Best lighthouse docs I can find: https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/configuration.md