Auth0 Automated Quickstarts Tests

This package intends to make it easier to run automated test against the Auth0 SPA and Web Quickstarts samples.

Requirements

This is a Node JS project that uses Casper JS to run the web automation tests.

• Python
• CasperJS: Run brew install casperjs to install the Casper JS binary. Running casperjs --version should work. Refer to the installation guide for more information.
• PhantomJS: Casper JS is based on Phantom JS and should be installed automatically as a dependency. Running phantomjs --version should work. If that doesn't happen, refer to the installation guide for more information.
• Docker Desktop: Refer to the installation guide for more information.

The samples must be "docker ready" and contain a exec.sh file with the docker build and docker run commands. This script is incompatible with the -it flags of the docker run command, so those are stripped from the file before the execution.

The main tenant credentials must belong to a STAGE region tenant. If you need to change the region, modify the casper/steps/01_create_tenant.js and casper/steps/02_switch_tenant.js files.

API

Start by requiring the package and creating a new QSTester instance.

var QSTester = require('auth0-quickstarts-tester');
var tester = new QSTester();

Running for a bunch of Quickstart samples

This function uses "Main Tenant" credentials with access to the STAGE region to create a "Test Tenant" with a new Client to run the tests against. The advantage of this scenario is that any default flags defined when a new Tenant or Client is created that could affect the behavior of a simple Authentication are also tested. The function is runAll(dashboardLogin, testParameters, repositories) function.

var dashboardLogin = {
  user: "an-existing-tenant-username-or-email",
  password: "the-tenants-password"
};
var testParameters = {
  url: "http://localhost:3000",
  user: "an-existing-user-username-or-email",
  password: "the-users-password"
};
var repositories = ["sample-one", "sample-two"];
tester.runAll(dashboardLogin, testParameters, repositories)
  .then(() => process.exit());

Pre-conditions:

• Samples must be "Dockerized" and contain an exec.sh file at the 01-Login path with the docker instructions to properly build and run the container.
• dashboardLogin defines an existing tenant's email and password used to create a new Tenant. The tenant must have access to the STAGE region.
• testParameters defines a URL where all the web apps are expected to be accessible, along with a valid username/email and password used to log in into the app.
• Each of the repositories in the array exist under the https://github.com/auth0-samples organization. The array accepts values of the following type as well:

{
  repository: "sample-repo-name",
  organization: "lbalmaceda",     //Defaults to "auth0-samples"
  branch: "dev",                  //Defaults to "master"
  path: "02-renew-authentication" //Defaults to "01-Login"
}

Steps followed:

1. Log in using the Main Tenant credentials.
2. Create a new Test Tenant using a random name in the Stage region.
3. Create a new client, connection and user on that tenant.
4. Download and unzip each sample listed in repositories.
5. Prepare the Docker container and run it. This is done by calling a script file located in the root of the sample project that builds the image from it's contents. The contents of the downloaded sample change on each execution as they depend on the tenant that was logged in at that moment.
6. When the web app is up and running, the log in tests are run against that app.
7. Stop the running containers. Beware! This step will stop all the running containers, not just the one created by this script.
8. Remove/Delete the sample folder.
9. Repeat 4-8 steps for as many repositories as required.
10. Log in again using the Main Tenant credentials.
11. Switch to the Test Tenant and delete it.

If something happens after creating the Test Tenant, the script will automatically run the teardown methods in order to leave the dashboard as it was at first.

Running a single Quickstart sample

Suppose you've already downloaded/cloned a sample like the auth0-javascript-samples one. You'd want to skip the Dashboard setup, the Quickstart download and the Dashboard teardown steps that are included in the run for all function. You do so by using the runOnPath(samplePath, testParameters) function.

var testParameters = {
  url: "http://localhost:3000",
  user: "an-existing-user-username-or-email",
  password: "the-users-password"
}
tester.runOnPath("./auth0-javascript-samples/01-Login", testParameters)
  .then(() => process.exit());

Pre-conditions:

• Sample must be "Dockerized" and contain an exec.sh file at the samplePath with the docker instructions to properly build and run the container.
• Sample client_id and domain values and any other required value has already been set with the details from a valid Auth0 Client. Follow the sample configuration guide to accomplish this.
• testParameters defines a URL where the web app is expected to be accessible, along with a valid username/email and password used to log in into the app.

FAQ

Q: The script execution fails.

R: Make sure that casperjs --version, phantomjs --version and docker --version return successfull messages. Also Docker daemon must be running.

Q: I don't see the Stage region enabled for my tenant.

R: Ask nicely if you can get it enabled or if you can get the credentials for an existing one. The Stage region is dedicated for testing purposes mainly, and it's were we should create this disposable "test tenants".

Q: Some Casper JS tests are failing because of a timeout.

R: This doesn't happen a lot. A delay of 10 secs is set for each test step. Retry.

Q: Some Casper JS tests are failing due to elements that are not found in the page.

R: Probably you're using a different "Hosted Login Page" script, or the CSS class of the components has changed. You'll have to edit the tests under casper/steps.

Q: The browser doesn't open when running the tests.

R: This is how Casper JS works. If you need help debugging, check the screenshots under the screenshot folder.

Q: The sample I want to test doesn't support Docker yet.

R: Feel free to fork the project and add it on a new branch.

Debugging

• The package outputs the stdout and stderr to a new file on the logs folder.
• The execution of CasperJS tests outputs screenshots on some of the steps.

Note that the folders above are located on the package root (node_modules/auth0-quickstarts-tester).