ctk-cypress-integration-test

POc for migration from Postman to Cypress for Gateway and Hierarchy Gateway APIs integration tests

Local setup

Prior to local execution, the JWKS secrets need to be imported to the local machine for all the environments that will be tested:

• You'll need to be on node version 16 or above.

• You'll need to run aws sso login to login to aws before running these tests. You will need at least doolittle-nonprod-team-role for DEV/QA/Staging, and doolittle-prod-team-role for PROD:

• Import the secret for the environment you are interesting on testing:

  • For DEV export CYPRESS_JWT_SECRET=$(aws secretsmanager get-secret-value --secret-id /ns/doolittle-dev/InternalJwksSecret --region us-west-2 | jq '.SecretString | fromjson | .keySet' | sed -e 's/^"//' -e 's/"$//')

  • For QA export CYPRESS_JWT_SECRET=$(aws secretsmanager get-secret-value --secret-id /ns/doolittle-qa/InternalJwksSecret --region us-west-2 | jq '.SecretString | fromjson | .keySet' | sed -e 's/^"//' -e 's/"$//')

  • Note: you may need to install jq to run above. For Mac, brew install jq is one way to install.

  • JWKS secrets rotate in AWS Secrets Manager roughly every two weeks or so. Each rotation will require the new secrets to be imported again.

• Run npm install

• Now generate latest support code for GraphQL queries by running npm run generate

Local environment

If the integration tests are to be executed against the local docker environment, the corresponding secret will be exceptionally accepted as an empty string (""). No setup is required for this case.

to execute locally, run npm run cy:run:qa

to execute locally using cypress interface, run npm run cy:open:qa

you can replace qa for dev in the commands above for the desire environment

the env variable can be local, dev, qa, staging, prod, as defined in cypress.config.js.

Best Practices

Terms

• Test Suite - One ts file that has at least one describe block and one or more it blocks.
• Test An it block within a Test Suite.

Writing a test

Each test should

1. Do setup. Set state and/or use the API to create test data on the server to create the starting state for that particular test
2. Execute API method under test.
3. Assert that the resulting response and backend state is what is expected.
4. Clean up test state or database state if either was changed.

Guidelines

We should always ensure that each test is independent of other tests.

Each test it block should not be dependent on any code or test data from any other it block. If you remove all the other it blocks in a suite and all the other suites, and run only your test, your test should still work properly. In addition, your test should not affect the results of any other test.

We should strongly prefer creating test data specifically for each test.

Use the APIs themselves to create test data in your test setup code in beforeEach or the it block itself. This allows our tests to be independent of the existing data that can be inadvertently changed by other tests or by users. Exceptions might be made due to execution time of test data creation. Next best approach is to create only within the test suite for all or a subset of tests.

We should strongly prefer to remove our test data and reset our test variables after every test

Use deprecate mutations to delete data, use TestVar.clear to clear all variables, etc so that every test sets the environment back to what it was before the test ran.

We should prefer using native expect() to make test assertions.

To make it clear to users what is being tested in a test, it is preferred to use native cypress assertion methods such as expect(). Exceptions might be made if you have complicated assertions that are reused across tests so you make a custom assertion method. This method should have the word "expect" within it (ex. expectQuestionToBeCompleted)to make clear it is making an assertion.

We should share saved variables and data across test suite blocks using TestVar methods.

When saving data to use across before, beforeEach, it, after, and afterEach blocks in a test suite, we should use TestVar.set and TestVar.get. This allows use to clearly identify variables we are saving for use between tests and easily reset them using TestVar.clear

We should strongly prefer creating test data unique to the running test

We should try to use unique values when querying for and asserting on returned data. We can use unique ids using generateUniqueId when we create test data via APIs and assert on the unique data in the tests.

We should always add message parameter to expect statements

We should always add the message parameter to our expect statements to make reading the test report clear. For example, expect(contentId).is.not.null will be reported in the test report as "Expected null to not be null" which doesn't make clear you are checking contentId. If you write expect(contentId, 'content id').to.be.null then its reported as "content id: expected null to not to be null" which is much clearer when debugging a test that has multiple expect statements. We should write short messages that make the test report easy to read and debug.