@userway/a11y-webdriverio
v0.0.8
Published
Accessibility checker for WebdriverIO
Downloads
92
Maintainers
Readme
UserWay App WebdriverIO
Intro
The @userway/a11y-webdriverio
is an NPM package designed to help you perform accessibility testing on your web pages. With it you can easily run static page analysis on a webpage and get a detailed report of accessibility violations based on WCAG guidelines and ACT rules.
// e2e/accessibility.test.js
const { browser } = require('@wdio/globals')
const { userwayAnalysis } = require('@userway/a11y-webdriverio')
;(async () => {
await browser.url('https://my_url.com', {})
const result = await userwayAnalysis(browser, {})
await browser.closeWindow()
})()
Prerequisites
WebdriverIO version 8.40.5
or higher
Distribution
UserWay App WebdriverIO is distributed as a zip-packaged NPM module and should be installed as an NPM dependency.
Installation
First, extract the provided userway-webdriverio-app.zip
inside of a separate
directory, like src/packages
:
src
└── packages
├── userway-app-webdriverio.zip
└── userway-app-webdriverio
Install userway-app-webdriverio
with npm install
:
npm install src/packages/userway-webdriverio-app
This adds @userway/a11y-webdriverio
to the dependencies in package.json
.
Setup
UserWay App WebdriverIO
external package and needs to be imported in webdriverio test.
Import @userway/a11y-webdriverio
to your WebdriverIO's test file and call userwayAnalysis
function:
const { userwayAnalysis } = require('@userway/a11y-webdriverio')
test('example test', async () => {
await browser.url('https://demo.evinced.com')
await userwayAnalysis(browser)
})
Types
If you are using TypeScript, add @userway/a11y-webdriverio
to types
section in your tsconfig.json:
{
"compilerOptions": {
"types": ["webdriverio", "@userway/a11y-webdriverio"]
}
}
If you are not using TypeScript, you can still have autocompletion available by adding type references to your tests:
/// <reference types="webdriverio" />
/// <reference types="@userway/a11y-webdriverio" />
// ↑ Add this at the top of your test
Usage
This example shows how to use UserWay App WebdriverIO to run static page analysis on a webpage:
;(async () => {
await browser.url(
'we need to deploy some demo page (Evinced demo: https://demo.evinced.com/)',
)
await userwayAnalysis(browser)
await browser.closeWindow()
})()
By default, UserWay App WebdriverIO scans the page for AA violations and asserts on the number of issues, but you can easily customize it's behavior. Only the first window, opened by browser.url is being scanned. Scanning of multiple windows on one browser is not possible.
API
userwayAnalysis(browser: Page, config: AnalyzeConfig): AnalysisResult
Runs static analysis on the current page and returns a result object that contains violations. By default asserts that the number of violations is equal to zero.
Default config:
userwayAnalysis(browser, {
strict: false,
level: 'AA',
})
To make manual assertion you can use strict: false
:
userwayAnalysis(browser, { strict: false }).then((res) => {
expect(res.violations).to.have.length(0)
})
Get access to the configuration object from results:
userwayAnalysis(browser, { strict: false }).then((res) => {
console.log(res.fullReport.config)
})
Full config:
type Config = {
excludeRules: string[] // Provide a list of rules that should be excluded
failInapplicable: boolean // *Maybe remove*
failIncomplete: boolean // *Maybe remove*
strict: boolean // Set to `true` if you don't want to manually process the result without automatic assertion
includeBestPractices: boolean // Set to `true` if you want to use best practices rules
includeExperimental: boolean // Set to `true` if you want to use experimental rules
includeRules: string[] // Provide a list of rules that should be included
level: 'A' | 'AA' | 'AAA' // Conformance level (`null` disables all A, AA and AAA rules)
onResult: (data: Result) => void // Hook that called when result is ready
reportPath: string // Provide path to folder where to store artifacts ( global configuration only ) Default is "uw-a11y-reports"
elementScreenshots: boolean // Set to `true` if you want save a screenshots of violations
includeIframes: boolean // Set to `true` if you want to include subdomains to analysis
ignoreSelectors: string[] // Specify selectors for elements that should be ignored. Default is ["data-userway-app-ignore"]
switchOff: boolean // Allows to turn off rules check without any modification of the tests
}
Examples:
// Save report and screenshots
userwayAnalysis(browser, {
elementScreenshots: true,
})
// Enable all rules
userwayAnalysis(browser, {
level: 'AAA',
includeBestPractices: true,
includeExperimental: true,
})
// Enable best practices and experimental rules only
userwayAnalysis(browser, {
level: null,
includeBestPractices: true,
includeExperimental: true,
})
// Enable rules from includeRules parameter only
userwayAnalysis(browser, {
level: null,
includeRules: ['duplicate-id', 'color-contrast'],
})
// Disable all rules
userwayAnalysis(browser, {
level: null,
})
// Manually assert
userwayAnalysis(browser, { strict: false }).then((res) => {
const { fullReport, violations } = res
// Log the full report
console.log(fullReport)
// Assert
expect(violations).to.have.length(0)
})
setupUserway(config: AnalyzeConfig): void
In order to avoid providing configuration for each launch of userwayAnalysis
function you can provide global configuration
The best option is to set global settings in WebdriverIO config or global setup file, but if you use setupUserway
please make sure that it's called only once before your tests
Note: Global configuration could be overridden by passing configuration directly to userwayAnalysis
in you tests
// webdriverio.config.js
const { setupUserway } = require('@userway/a11y-webdriverio')
setupUserway({
reportPath: 'uw-a11y-reports',
})
module.exports = {
// configuration details
}
;(async () => {
await browser.url('http://localhost:5000')
await userwayAnalysis(browser, {
printViolationsTable: true,
elementScreenshots: true,
})
await browser.close()
})()
Configuration override
Global configuration can be overridden, the rule of thumb is that specific method override more general definition
For example:
// webdriverio.config.js
const { setupUserway } = require('@userway/a11y-webdriverio')
setupUserway({
level: 'AA',
ignoreUrls: [/localhost:3000/],
})
// some webdriverio test file
await userwayAnalysis(browser, {
level: 'A',
ignoreUrls: [/home/],
})
// ends up with such configuration
// {
// level: 'A',
// ignoreUrls: [/home/, /localhost:3000/]
// }
Note that types like arrays and objects won't be overridden, they will be merged with previous defined values
saveReportArtifacts(config: SaveReportConfig): void
Save violation report in manual control mode. It's handy when you need to save report conditionally, based on violations reported or your own custom logic.
saveReportArtifacts
depends on report from userwayAnalysis
and requires violations
and reportPath
data to be provided.
Note: outputPath
from userwayAnalysis
should be passed to saveReportArtifacts
in all cases to make test result separation work correct.
Output format could be provided by configuring format
parameters.
In order to have screenshots attached to report, you need to provide screenshotsMeta
to saveReportArtifacts
This data can be extracted from userwayAnalysis
if screenshots
parameter passed in the method config.
Example:
userwayAnalysis(browser, {
strict: false,
elementScreenshots: true,
}).then((res) => {
const { fullReport, violations, screenshotsMeta, outputPath, meta } = res
// Save JSON report
saveReportArtifacts({
violations,
saveReport: 'json',
reportPath: outputPath,
meta,
})
// Save CSV report with custom report path
saveReportArtifacts({ violations, saveReport: 'csv', reportPath: outputPath })
// Save HTML report with screenshots of violated elements
saveReportArtifacts({
violations,
saveReport: 'html',
reportPath: outputPath,
screenshotsMeta,
})
})
Config
level
Specify the conformance level. Possible values: A
, AA
, AAA
. Default is AA
.
ignoreSelectors
List of selectors for elements to ignore. Accepts an array of strings. Default selector is "data-userway-app-ignore"
Note: use "ignoreSelectors" only to exclude specific elements, avoid using it in root element like body, head, html ( for such elements you can manually exclude related rules ) To see list of all rules related to html or body elements refer to "Rules" section.
ignoreUrls
Use this parameter if you need to skip a tests that open specific URLs.
Accepts an array of valid RegExp. Default is undefined
Examples:
userwayAnalysis({
ignoreUrls: [/localhost:3000/, /example.com/],
// Ignore a specific domain
})
userwayAnalysis({
ignoreUrls: [/home/, /http:/],
// Ignore any url that contains specified string
})
userwayAnalysis({
ignoreUrls: [/home/, /settings\/privacy/],
// Ignore a specific path
})
userwayAnalysis({
ignoreUrls: [/home\/*/, /settings\/*/],
// Ignore a specific path and any of its subdirectories or path segments
})
reportPath
Specify custom path where to store accessibility reports and artifacts. Could be specified only in global configuration. Default is uw-a11y-reports
screenshots
Specify whether screenshots should be saved.
Note: Enabling the screenshots feature may have an impact on performance.
includeRules
List of rules to include. Accepts an array of rule IDs. Default is []
.
excludeRules
List of rules to exclude. Accepts an array of rule IDs. Default is []
.
Note, that this parameter has higher priority then includeRules
.
It means that same rule provided to includeRules
and excludeRules
parameters will be ignored.
includeBestPractices
Specify whether to include best practices rules. Default is false
.
includeExperimental
Specify whether to include experimental rules. Default is false
.
strict
Specify whether to make assertion on the number of accessibility violations. Default is false
includeIframes
Specify whether to include subdocuments into analysis. Default is false
.
Rules
These kind of rules are reviewing html or body element, and if you want to ignore these elements - exclude the rules that check them.
- aria-hidden-body
- bypass
- bypass-heading
- bypass-landmark
- bypass-move-focus
- css-orientation-lock
- document-title
- html-has-lang
- html-lang-valid
- html-xml-lang-mismatch
- meta-viewport
- meta-viewport-large
- page-has-heading-one
- valid-lang
Reports
Document boundary selector
This selector >>>
is used to represent the relation between a document and its subdocument like
document - iframe, document - Shadow DOM, iframe - iframe, etc. In the report it means that violated element located inside a subdocument.
OutputType
The outputType
field in violation reports can contain group
as a value, and it means that violated elements grouped by issue root cause.
It is very handy for violations with duplicate values ( like: ids, alt attributes, landmarks )
Examples for duplicate-id
rule violation:
<div>
<span id="one">goes to the first group</span>
<p id="two">goes to the second group</p>
<span id="two">also goes to the second group</span>
<p id="one">goes to the first group</p>
</div>
In such case extra field issuesGroup
added to report and contains grouped elements by ID attribute which was duplicated.
{
issuesGroup: {
one: [issue('span#one'), issue('p#one')],
two: [issue('p#two'), issue('span#two')],
}
}
Report path customization
There is a parameter reportPath
from the config that can be used to customize the output path for accessibility reporting.
Also besides the parameter you can define environment variable USERWAY_CA_REPORT_PATH=your_custom_path
before webdriverio tests execution:
USERWAY_CA_REPORT_PATH=custom-path node ./dev/webdriverio.test.js
The path should be relative to the project directory from where webdriverio
command is executed.
In the following example folder uw-a11y-reports
will be created in the same level where webdriverio
command is executed ( most probably with the package.json file )
The reports folder will contain HTML file with output and another folder with related screenshots ( because elementScreenshots: true
passed to static analysis function )
To provide easier access to executed tests results their reports in the folder grouped by execution timestamp.
Report structure
Reports are saved to a folder specified by reportPath
or the default one uw-a11y-reports
folder from the project root. The reports folder has the following structure:
reports
- folder with accessibility reports that contain violation information and references to other artifacts.
pages
- folder with original HTML documentation of web pages that have been tested
screenshots
- folder with screenshots of found accessibility violations ( only with enabled screenshots feature )
// webdriverio.config.js
const { setupUserway } = require('@userway/a11y-webdriverio')
setupUserway({
reportPath: 'uw-a11y-reports',
})
await userwayAnalysis(browser, {
elementScreenshots: true,
})
Error messages and recommendations
Each violation in the report contains two fields errorMessage
and recommendation
that refer to WCAG standard and provided for better understanding of the violation's root cause with the guidelines of how to fix it.
Violation variations
Some violations might have a few root causes. For example, the rule image-alt
can be violated by missed alt
attribute or by having it unfilled with an empty string.
The field variant
in the violated element selector points to related information described inerrorMessage
and recommendation
that helps to identify the root cause and appropriate fixes more granularly.
Global Switch
Global Switch allows to switch off or switch on accessibility analysis. It could be needed, for example, while debugging tests in your local environment with or without static analysis tool.
There are two options to switch functionality off:
- Provide
switchOff: true
config parameter to global configuration bysetupUserway
method call.
// webdriverio.config.js
const { setupUserway } = require('@userway/a11y-webdriverio')
setupUserway({
switchOff: true,
})
module.exports = {
// configuration details
}
Note: switchOff
parameter can be also specified for userwayAnalysis
method.
- Define environment variable
USERWAY_CA_SWITCH_OFF=true
before playwright tests execution
USERWAY_CA_SWITCH_OFF=false node ./dev/webdriverio.test.js
Background Mode
Environment variable USERWAY_WEBDRIVERIO_MODE
determines in which mode the plugin will work.
You can choose mode of your own choice when passing the config, but be aware that it is limited by the USERWAY_WEBDRIVERIO_MODE
you have set.
Options:
all - default
background
manual