Netlify E2E Tests

This package is responsible for defining the Netlify End-to-End tests.

The tests are distributed as npm package @netlify/e2e along the test helpers to create new test fixtures. This is done so that other repositories like @netlify/build can leverage the same tests and test pipeline with minimal effor.

Adding or changing a test must be done inside @netlify/e2e and it needs to be released.

How it works:

The tests are defined under packages/e2e-tests/tests folder using the Playwright test executor.

The fixtures are defined in the tests/fixtures.ts file. There can be 2 type of test setups used inside the fixtures file:

1. Test against a site that is already setup on Netlify. (E2E sites should have the E2E label on it and should be inside the Netlify Testing organization)
2. Create a new site from a GitHub repository.

   caveat: This test requires that the [email protected] user has permissions to add deploy keys to the GitHub repository.

Adding a new test:

Update the tests/fixtures.ts file by adding a new fixture:

export const fixtureFactories = {
  // ... other fixtures
  astroPlatformStarter: {
    siteId: 'fb957dc7-114c-4d5f-bd1c-a9d856204233', // The Netlify site ID that should be used to perform the test
  },
}

Or from a GitHub repository:

export const fixtureFactories = {
  // ... other fixtures
  astroBasic: {
    repoPath: 'netlify/astro',
    publishDir: 'examples/basics/dist',
    buildCommand: 'turbo run build --filter @example/basics',
    packagePath: 'examples/basics',
  },
}

The test can be written then in its spec file where the fixture is imported:

Assure that the test function is imported from ./fixtures.js > import { test } from './fixtures.js'

import { expect } from '@playwright/test'
import { test } from './fixtures.js'

test('Simple Astro page', async ({ page, simpleAstro }) => {
  await page.goto(simpleAstro.site.url)

  await expect(page).toHaveTitle(/Welcome to Astro./)
  await expect(page.locator('h1')).toHaveText('Welcome to Astro')
})

Local test development

For local Development you need to set the following environment variables inside the .env file:

# In Dept-Engineering 1Password: [email protected] Netlify PAT
NETLIFY_TOKEN=<a netlify pat>
# In Dept-Engineering 1Password: GitHub - Netlify Bot
GITHUB_TOKEN=<a github pat that can write to repos and add deploy keys>


# Only needed for testing the image building part:
# In Dept-Engineering 1 Password: [email protected] CIRCLECI Personal Access Token
CIRCLECI_TOKEN=<a pat for triggering new pipelines>

Inside a fixture you can provide a buildImage property that points to a already pre-built image to avoid having to rebuild that image all the time:

export const fixtureFactories = {
  // ... other fixtures
  astroPlatformStarter: {
    siteId: 'fb957dc7-114c-4d5f-bd1c-a9d856204233', // The netlify site id that should be used to perform the test
    buildImage: '92f1cf525c009b7cfa413117256b190998446b98-focal-e2e',
  },
}

Then you can just run:

pnpm --filter @netlify/e2e run e2e

Often for debugging you want to inspect the deploy logs or see the build configuration. To do so you can run the command with PRESERVE_SITE=true pnpm e2e to prevent deleting the Netlify Site.