dssrv-testee
v0.3.3
Published
Automated cross-browser testing made easy.
Downloads
6
Maintainers
Readme
Testee
Run your QUnit, Mocha or Jasmine tests from the command line with any browser.
- Runs on all browsers (supporting SocketIO)
- Many output formats
- CI integration
- BrowserStack support
- Code coverage
- GruntJS Task
- Remote URL testing
For questions or discussion checkout our forums.
Getting started
You need NodeJS installed in order to use Testee which makes the installation as easy as:
npm install -g testee
cd
into the main folder of your JavaScript project and run testee
with your QUnit, Mocha or Jasmine
test HTML page:
testee tests/qunit.html
Command line testing
The default browser is PhantomJS, just make sure you have it installed anywhere on your system.
To run with different local browsers (e.g. Firefox and Safari) use:
testee tests/qunit.html --browsers firefox,safari
Or run multiple tests in multiple browsers:
testee tests/unit.html tests/components.html --browsers firefox,safari
Note that the browser you are using for testing shouldn't be already running (except for PhantomJS which can be started multiple times).
Command line options
On the command line, you have the following options available:
-h
,--help
: output usage information-V
,--version
: output the version number-b
,--browsers
[name]
: A comma separated list of browsers you want to run (default:phantom
)-R
,--root [path|URL]
: The server root path or URL the files are relative to-p
,--port
[port]
: The port to run the server on (default:3996
)-r
,--reporter
[name]
: The name of the reporter to use (default:Dot
)-c
,--config
[file]
: Use this JSON configuration file (can be overwritten by command line options)--timeout
[seconds]
: The per test timeout (in seconds)--delay
[ms]
: When running multiple tests, the time to wait for the browser to shut down before starting it with a new page.-s
,--server
: Only run the server--coverage
: Enable code coverage
Examples
Run test.html
in the /var/www/app/
folder using Safari:
testee test.html --root /var/www/app/ --browsers safari
Run the online Underscore QUnit tests in Phantom and Firefox and output code coverage statistics:
testee test/index.html --root http://underscorejs.org --browsers phantom,firefox --coverage
Run tests/qunit.html
with PhantomJS from the current folder, use port 8080
instead of 3996
and
use the Spec
reporter which prints more detailed test results:
testee tests/qunit.html --port 8080 --reporter Spec
Run tests/mocha.html
using testee.json
as the configuration file (see the configuration API
for how this file should look like):
testee tests/mocha.html -c testee.json
Run tests/jasmine.html
using Google Chrome Canary:
testee tests/jasmine.html --browsers canary
CI integration
Because Testee allows to use different reporters for the test result output it is easy to obtain XUnit
style XML files that integrate with CI servers like Jenkins. Just use the XUnit
reporter and write the output into a file. The following example runs tests/qunit.html
in Firefox and writes
the result XML into testresults.xml
:
testee test/index.html --browsers firefox --reporter XUnit > testresults.xml
You can get more information about the available reporters in the Reporters section.
Debugging
Testee uses the Node debug module. Detailed debugging information can be enabled
in any environment (command line, Grunt, programatically) by setting the DEBUG
environment variable to testee:*
:
DEBUG=testee:* testee --browsers canary tests/jasmine.html
Configuration API
The following sections describe the available options for
- the JSON configuration file if you run Testee from the command line using the
-c
or--config
option - the GruntJS task task configuration
- programmatic usage with NodeJS
Default configuration
Any options will be merged with the following default configuration:
{
port: 3996,
root: process.cwd(),
reporter: 'Dot',
timeout: 120,
delay: 1000,
tunnel: {
type: 'local'
},
launch: {
type: 'local'
}
}
General settings
root
{String}
Every time when running a test, Testee will start a static file server, by default in the current folder.
That way, any test HTML file you reference will be loaded properly. The root
option allows you to change
the root path of the static fileserver.
port
{Integer}
The port for the static file server to start on. This will also be used by Localhost tunneling services. The default is 3996
.
timeout
{Integer}
The time (in seconds) to wait for a test page to report back and after which an error will be thrown.
The default is 2 minutes. This timeout might, for example, occurr when the given file doesn't exist the
browser didn't start or the localhost tunnel isn't running.
delay
{Integer}
Multiple test files will be run in sequence on the same browser. This option sets the delay (in ms) between
launching the browser again with the same file.
Reporters
The reporter
option allows you to use all of the console reporters included in the
Mocha testing library.
Launching browsers
The launch
and browsers
options are used to set the environment and the browsers you want
to start. Launchpad is the browser launcher library used
by Testee which allows you to start most locally installed browsers as well as Browserstack workers
and even browsers on remote systems running the Launchpad Server.
The most common case will be launching a local browser (which is also the default) with no settings:
{
"browsers" : [ "firefox", "safari" ]
}
Browserstack
Browserstack hosts virtual machines running specific versions of web browsers. It is an extremely useful tool for cross-browser testing running a remote desktop in you browser. To use BrowserStack via the configuration API you need to provide a username and password:
{
"launch": {
"type": "browserstack",
"username": "your browserstack username",
"password": "your browserstack password",
"version": "browserstack API version (recommended: 2)"
}
}
To start a worker, you must provide a valid browser object to the browser
option:
{
browsers: [{
os: "win",
browser: "ie",
version: 8.0
}, {
os: "win",
browser: "ie",
version: 11.0
}]
}
An example configuration that runs your tests on an iPad Mini and Samsung Galaxy S3 emulator using BrowserStack in a CI environment (outputting XUnit logs) could look like this:
{
"reporter" : "XUnit",
"tunnel": {
"type": "browserstack",
"key": "your browserstack key"
},
"launch": {
"type": "browserstack",
"username": "your browserstack username",
"password": "your browserstack password",
"version": 2
},
"browsers": [{
"os": "ios",
"device": "iPad Mini",
"version": 6.0
}, {
"os": "android",
"device": "Samsung Galaxy S III",
"version": "4.1"
}]
}
}
Code Coverage
The coverage
options is used to instrument and report code coverage using Istanbul. There are several options available for configuring how code coverage is reported:
reporters
{Array}
The type of reporter(s) to use. Multiple formats are available, including text
and html
.
dir
{String}
The directory where the coverage data should be written. text
reports will be written to the console.
ignore
{Array}
List of regex patterns to match files that should be ignored by coverage instrumentation and reporting.
Localhost tunneling
A localhost tunneling service makes your local system available to the outside world. This is great if you want to run tests on another system which can't easily reach your local machine. Testee relies on localhost tunneling services especially for giving Browserstack workers an endpoint to communicate with.
Testee uses the Miner package to provide localhost tunneling which makes it possible to use any of the services Miner currently supports (LocalTunnel, Pagekite and Browserstack). Localtunnel doesn't need any configuration at all and will install itself if you have Ruby available.
If you would like to use Pagekite you need to set it up with your username and then pass it to the launch
option
like this:
"launch" : {
"type" : "pagekite",
"username" : "pagekit user"
}
It is also possible to use the Browserstack tunnel API which you have to provide with your command line tunnel API key:
"launch" : {
"type" : "browserstack",
"key" : "your command line tunnel API key"
}
For all available tunneling services and options follow up in the Miner documentation.
Grunt task
Testee comes with a Grunt task that takes the same options as described above (src
should be set to your source test files).
The following is an example that runs test/index.html
with public/
as the root folder and Spec
as the reporter in either
- PhantomJS
- PhantomJS with code coverage ignoring the
bower_components
andtest/
folder - On Browserstack on iPad Mini and Samsung Galaxy S3 with
BROWSERSTACK_USER
andBROWSERSTACK_PASSWORD
taken from environment variables
module.exports = function(grunt) {
grunt.initConfig({
testee: {
options: {
root: 'public',
reporter: 'Spec'
},
coverage: {
options: {
browsers: ['phantom'],
coverage: {
dir: 'test/coverage/',
reporters: ['text', 'html'],
ignore: ['bower_components/', 'test/']
}
},
src: ['test/index.html']
},
browserstack: {
options: {
timeout: 600,
tunnel: {
type: 'browserstack',
key: process.env.BROWSERSTACK_PASSWORD
},
launch: {
type: 'browserstack',
username: process.env.BROWSERSTACK_USER,
password: process.env.BROWSERSTACK_PASSWORD,
version: 2
},
browsers: [{
"os": "ios",
"device": "iPad Mini",
"version": 6.0
}, {
"os": "android",
"device": "Samsung Galaxy S III",
"version": "4.1"
}]
},
src: ['test/index.html']
}
}
});
grunt.loadNpmTasks('testee');
};
Programmatic Use
You can also use Testee programmatically from gulp or another Node script.
gulp.task('test', function() {
return testee.test(['test/test.html'], 'phantom', {
reporter: 'Spec'
});
});
Client side configuration
In most cases there is no need to change your actual test code.
One exception is when you load your testing library using an asynchronous client side loader like Steal or RequireJS because Testee won't know which library adapters to attach. In this case, you need to call Testee.init()
manually once the test library is loaded:
<script type="text/javascript">
define(['qunit'], function() {
// Needs to check because it will only be available
// when running the test with Testee
if(window.Testee) {
window.Testee.init();
}
QUnit.start();
});
</script>
In some testing environments, reporting test progress via REST may work better than socket.io. In this case, you specify a provider.type
or 'rest'
:
<script type="text/javascript">
window.Testee = {
provider: {
type: 'rest'
}
};
</script>