Lighthouse Audit Service

A service meant to help you run, schedule, store, and monitor Lighthouse reports over time. The API is built with Backstage in mind, but can be used without!

Usage

With Docker

The simplest way to deploy the app is with our image on Docker Hub.

docker run spotify/lighthouse-audit-service:latest

Be sure to see "Configuring Postgres" - you will likely need to configure the Postgres credentials, even when trying the app out locally. A list of supported environment variables:

• LAS_PORT: which port to run the service on
• LAS_CORS: if true, enables the cors express middleware.
• all environment variables from pg, which should be used to set credentials for accessing the db.

With Docker Compose

A simple way to trial this tool is with the following docker compose file which spins up a postgres container and connects it with the lighthouse image. This would not be a great way to run this in a production environment, but a fast way to test out this tool and see if it meets your needs.

# docker-compose.yml
version: '3.1'

services:

  db:
    image: postgres:latest
    restart: always
    environment:
      POSTGRES_USER: dbuser
      POSTGRES_PASSWORD: example
  
  lighthouse:
    image: spotify/lighthouse-audit-service:latest
    environment:
      PGHOST: db
      PGUSER: dbuser
      PGPASSWORD: example
      LAS_PORT: 4008
    ports:
      - "4008:4008"

As an npm package

Install the project:

yarn add @spotify/lighthouse-audit-service

Then, you may either start up the server as a standalone process:

import { startServer } from '@spotify/lighthouse-audit-service';

startServer({
  port: 8080,
  cors: true,
  postgresConfig: {
    db: 'postgres',
    database: 'mydb',
    password: 'secretpassword',
    port: 3211,
  },
});

As an express app

You may nest the express app that the lighthouse-audit-service exposes inside of another express app as a subapp by using getApp and app.use():

import express from 'express';
import { getApp as getLighthouseAuditServerApp } from '@spotify/lighthouse-audit-service';

async function startup() {
  const app = express();
  const lighthouseServer = await getLighthouseAuditServerApp();

  app.use('/lighthouse', lighthouseServer);

  const server = app.listen(4000, () => {
    console.log(
      'listening on 4000 - http://localhost:4000/lighthouse/v1/websites',
    );
  });

  // be sure to `.get()` the connection and end it as you close your custom server.
  server.on('close', () => {
    lighthouseServer.get('connection').end();
  });
}

startup();

Configuring Postgres

You will need a Postgres database for lighthouse-audit-service to use to manage the stored audits. The database will be configured on app startup, so you need only initialize an empty database and provide credentials to lighthouse-audit-service.

You can set the Postgres credentials up either by setting environment variables, which will be interpreted by pg:

PGUSER=dbuser \
PGHOST=database.server.com \
PGPASSWORD=secretpassword \
PGDATABASE=mydb \
PGPORT=3211 yarn start

..or, by passing the config in programmatically as postgresConfig:

import { startServer } from '@spotify/lighthouse-audit-service';

startServer({
  port: 8080,
  cors: true,
  postgresConfig: {
    database: 'mydb',
    host: 'my.db.host',
    user: 'dbuser',
    password: 'secretpassword',
    port: 3211,
  },
});

Both startServer and getApp support this. Further, both of these methods support optionally passing a pg client as an optional second argument.

import { Pool } from 'pg';
const conn = new Pool();
startServer({}, conn);

API

We offer a REST API, as well as some programmatic ways, to interact with lighthouse-audits-service.

REST

We are currently seeking contributions on documenting the API in a sustainable way (aka with Swagger/OpenAPI, preferably generated). For now, the REST API includes:

Audit routes

• GET /v1/audits - list of all audits run
• GET /v1/audits/:auditId - get an audit, either as HTML or JSON depending on the Accept header of the request.
• POST /v1/audits - trigger a new audit
  • expected JSON payload:
    • url: string - url to audit
    • options - all optional
      • awaitAuditCompleted: boolean - makes awaiting triggerAudit wait until the audit has completed. By default, the audit runs in the background.
      • upTimeout: number - time in ms to wait for your site to be up (default 30000). We test that your URL is reachable before triggering Lighthouse (useful if this Lighthouse test will run for an ephemeral URL).
      • chromePort: number - chrome port for puppeteer to use
      • chromePath: string - chrome path for puppeteer to use
      • lighthouseConfig: LighthouseConfig - custom Lighthouse config to be used when running the audit.
• DELETE /v1/audits/:auditId - delete an audit

Website routes

• GET /v1/websites - list of audits grouped by url
• GET /v1/websites/:websiteUrl - get the audits associated with this url. be sure to uri encode that url!
• GET /v1/audits/:auditId/website - get the group of audits associated with the url used for this audit.

Programatic

All of the API methods exposed on REST are also exposed programmatically.

Server

• startServer(options?, conn?) - start REST server
• getApp(options?, conn?) - return express app, ready to be started

Audit methods

• getAudits(conn, listOptions?) - list of all audits run
  • listOptions (all optional):
    • limit and offset for pagination
    • where: SQLStatement - using sql-template-strings, create a custom WHERE to inject into the query.
• getAudit(conn, auditId) - retrieve an Audit by id.
• triggerAudit(conn, url, options?) - trigger a new audit.
  • options (all optional):
    • awaitAuditCompleted: boolean - makes awaiting triggerAudit wait until the audit has completed. By default, the audit runs in the background.
    • upTimeout: number - time in ms to wait for your site to be up (default 30000). We test that your URL is reachable before triggering Lighthouse (useful if this Lighthouse test will run for an ephemeral URL).
    • chromePort: number - chrome port for puppeteer to use
    • chromePath: string - chrome path for puppeteer to use
    • lighthouseConfig: LighthouseConfig - custom Lighthouse config to be used when running the audit.
• deleteAudit(conn, auditId) - delete an Audit from the DB.
• Audit - class used by the Audit API.
  • properties:
    • url: string: the original url when the audit was requested
    • status: string: the status of the audit; COMPLETED, FAILED, or RUNNING
    • timeCreated: string: ISO-8601 time when the audit was created
    • timeCompleted: string?: nullable ISO 8601 time when the audit was completed, if it has completed
    • report: LHR?: nullable LHR object which contains the full Lighthouse audit.
      • rendering the HTML for an LHR can be done programmatically by the Lighthouse package. The lighthouse-audit-service REST API does this automatically.
    • categories: LHRCategories: nullable map of categories, stripped down to only include the scores. useful for lists of audits, when trying to keep the payload to a reasonable size.

Website methods

• getWebsites(conn, listOptions?) - list of audits grouped by url
  • listOptions (all optional):
    • limit and offset for pagination
    • where: SQLStatement - using sql-template-strings, create a custom WHERE to inject into the query.
• getWebsiteByAuditId(conn, auditId) - get Website associated with this audit.
• getWebsiteByUrl() - get Website associated with this url.
• Website - class used by the Website API.
  • properties:
    • url: string: the audited url
    • audits: Audit[]: list of Audits for that URL.

Contributing

This project adheres to the Open Code of Conduct. By participating, you are expected to honor this code.

Publish should occur on merge using web-scripts and semantic-release. Please use conventional commits to signal what the new version should be.

External contributions and issues are welcome!

License

Copyright 2020 Spotify AB.

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0