AQUA: Automated QUality Analysis

About

AQUA improves code quality by surfacing actionable metrics about JavaScript, CSS and C#.

AQUA raises the visibility of code quality and increases awareness within teams by giving immediate feedback about code smells before they become technical debt.

You are free to structure your projects anyway you like, and still take full advantage of AQUA. Simply create an aqua.project.json file that documents where things are.

Philosophy

The goal of AQUA is to improve code quality, without dictating a strict project structure. Ramp-up gradually by only configuring the the parts of AQUA you want to use. Not ready for end-to-end tests yet? Simply don't configure them. Don't want to type check your JavaScript code? Just leave out the "types" section. Even if you just start by linting your code, that will be a great step in the right direction!

While nothing is stopping you from simply editing a JavaScript file and committing it to source control; doing so will cause you to miss out on: linting, cyclomatic complexity analysis, type checking, running both unit and end-to-end tests, as well as documentation generation, and code coverage reports.

Features

JavaScript

• Lint Source Code for Syntax Errors and Anti-Patterns
• Analyze Code Against Complexity Thresholds
• Type Checking Source Code
• Enforce Minimum Typed Percentage Thresholds
• Advanced Minification
• Unit Test Runner
• Code Coverage Reports
• End-to-end Test Runner
• Documentation Generation
• Enforce Test Coverage Thresholds
• Graph CommonJS and AMD module dependencies (TODO)
• Automatically fix lint errors non-destructively (TODO)

CSS

TODO: pull requests welcome :)

• Lint StyleSheets for Syntax Errors and Anti-Patterns

C#

TODO: pull requests welcome :)

• Lint Source Code for Syntax Errors and Anti-Patterns (via FxCop)
• Analyze Code Against Complexity Thresholds (via Code Metrics PowerTool)
• Unit Test Runner
• Code Coverage Reports
• End-to-end Test Runner
• Documentation Generation

Environments

AQUA is designed to play well with the following types of projects.

• Web Projects (including AMD/RequireJS)
• Node.js Projects

Setup

You'll need to have node.js v10+ installed.

1. navigate to your project root
2. run npm install aqua -save-dev (this will take a few minutes)
3. install gulp globally npm install -g gulp

Workflow

AQUA uses Gulp to orchestrate the following suite of NPM Modules.

• JSHint: to lint the code & standardize syntax
• Complexity-Report: for complexity analysis
• Karma: to run the unit tests
• Closure Compiler: for type checking & minification
• Protractor: to run the integration tests
• JSDoc: to generate documentation

The Node modules above can the run individually (see tasks below) or chained together.

Run from the Command Line

AQUA dynamically creates gulp tasks for you, based on what is configured in your aqua.project.json file. Tasks are namespaced with all projects id in your config file. Any tasks that are not configured for all projects are simply skipped.

| task | does | |----------------------------------------------------------------------------------|------------------------------------------------------------------------------| |gulp | List all possible tasks for all projects | |gulp ? | List all possible tasks for all projects | |gulp {id} | Run all project tasks in the correct order | |gulp {id}-? | List all possible tasks for all projects | |gulp {id}-all | Run all project tasks in the correct order | |gulp {id}-lint | Lint the source code for syntax issues and anti-patterns | |gulp {id}-lint-fix | Automatically fix lint errors non-destructively | |gulp {id}-gpa | Analyze the source against complexity thresholds | |gulp {id}-unit | Run unit tests against the source code | |gulp {id}-chk | Type check the source code | |gulp {id}-min | Minify the source code and create source map | |gulp {id}-e2e | Run end-to-end tests against the minified code | |gulp {id}-doc | Generate documentation from code annotations | |gulp {id}-wch | Watch for file changes and automatically run tasks |

Console Customization

Avoid a lot needless typing! You can create a shortcut that will open the command prompt at any folder you want.

Use the steps below for Windows.

1. Right-click on your desktop and choose New > Shortcut
2. For the location paste in (with the folder you want to open): C:\Windows\System32\cmd.exe /K "cd /d [Drive/Folder/You/Want/To/Open]"
3. Click Next, Set a Name and Click Finish
4. Leave it on your desktop or drag it to your Taskbar to pin it there.

Also now that you have a shortcut you can customize your command window using Right-click > Properties:

• Change the font used to something more readable
• Decrease the font size to show more information
• Increase the window size to reduce line wrapping
• Increase the screen buffer to scroll farther back in the task history

Canceling a Task

So you ran gulp aqa-wch and you want to stop watching for changes to all projects source files. Or maybe you accidentally typed gulp aqa-e2e and don't want to wait for all the end-to-end tests to complete? Pressing Ctrl+c will pause the task and ask if you want to Terminate batch job (Y/N)? typing y enter will cancel the task.

Quiet Mode

If there is any issues found by the quality checks such as a lint error or failing unit test the system will beep, unless --shh is included. For example gulp aqa-unit --shh

Run from Chrome Browser

• Download the Gulp Devtools extension for Chrome Developer Tools from the Chrome Web Store.
• Gun npm install gulp-devtools -g
• Run gulp-devtools from the your project root folder (where your gulpfile.js is located)
• Open Chrome Dev tools, find the Gulp tab. The AQUA/Gulp tasks are now accessible from Chrome.

Troubleshooting

During install some of you may encounter some issues.

Most issues can be solved by one of the following tips, but if are unable to find a solution feel free to an issue via the repository issue tracker.

Update NPM

Sometimes you may find there is a weird error during install like npm's Error: ENOENT. Usually updating those tools to the latest version solves the issue.

• Updating NPM: npm update -g npm

Cleaning NPM cache

NPM has a caching system for holding packages that you already installed. We found that often cleaning the cache solves some issues.

• NPM Clean Cache: npm cache clean

Testing

To improve efficiency and reduce cognitive load, all tests are written in JavaScipt, using a single testing framework (Jasmine). This means you can use the same assertion patterns and mocking strategies whether you're writing end-to-end tests for a web project or unit tests for your Node.js project.

Unit Tests

Unit tests are written in Jasmine and for web projects are run with Karma. Unit tests also have access to the Jasmine JQuery custom matchers and fixture loaders for HTML, CSS and JSON.

Debugging

Unit tests can be run from the commandline with gulp aqa-unit or in your favorite browser. Debugging can be done by setting break points in the source code, using console.log and debugger; statements.

End-to-end Tests

Page Objects

End to end testing (aka Integration Testing) should be organized using the PageObject Pattern and run by the Protractor API.

Page Objects can be thought of as facing in two directions simultaneously. Facing towards the developer of a test, they represent the services offered by a particular page. Facing away from the developer, they should be the only thing that has a deep knowledge of the structure of the HTML of a page (or part of a page) It's simplest to think of the methods on a Page Object as offering the "services" that a page offers rather than exposing the details and mechanics of the page. As an example, think of the inbox of any web-based email system. Amongst the services that it offers are typically the ability to compose a new email, to choose to read a single email, and to list the subject lines of the emails in the inbox. How these are implemented shouldn't matter to the test.

Page Object Pattern Goals

• The public methods represent the services that the page offers
• Try not to expose the internals of the page
• Generally don't make assertions
• Methods return other PageObjects
• Need not represent an entire page
• Different results for the same action are modelled as different methods

Debugging

You can pause end-to-end tests for debugging by using browser.debugger() for more, see documentation on Debugging E2E Tests.

Configuring AQUA

Patterns

The AQUA config uses glob patterns similar to those in the various Unix shells to find and filter your files. Glob patterns resemble regular expressions somewhat, but have a much simpler syntax. The following character sequences have special meaning within a glob pattern:

• ! negates the pattern
• ? matches any one character
• * matches any number of characters
• {!glob} Matches anything that does not match glob
• {a,b,c} matches any one of a, b or c
• [abc] matches any character in the set a, b or c
• [^abc] matches any character not in the set a, b or c
• [a-z] matches any character in the range a to z, inclusive. A leading or trailing dash will be interpreted literally

While it's possible to manually include each set of files individually, it's encouraged that you use convention over configuration. For example say you want all your unit tests in a folder called specs, great, then use a wildcard pattern (specs/*.js) and the correct files will automatically be included as they are added or even renamed over time. All config files paths must be relative to where AQUA is run from.

aqua.config.json

Used to configure AQUA settings that apply to all of your projects

• docs: Optional Object; all settings related to generating documentation
  • dir: Required String; Where to put the generated documentation files for each project
  • css: Optional String; location of a stylesheet file to include into all documentation files
  • js: Optional String; location of a javascript file to include into all documentation files
  • theme: Optional String; the name of the docstap theme you want to use
• coverage: Optional Object; all settings related to generating code coverage reports
  • dir: Required String; Where to put the generated coverage reports for each project
• testing: Optional Object; all settings related to testing all projects
  • framework: Required String; Type of testing framework to run the tests in
  • web: Optional String; location of the Karma config file used to unit test web projects
  • e2e: Optional String; location of the protractor config file used to test web projects
  • node: Optional String; location of the config file used to unit test Node.js projects
• thresholds: Optional Object; all settings related to testing all projects
  • complexity: Optional Object; configure complexity thresholds for all projects
    • cyclomatic: Required Number; cyclomatic complexity threshold
    • halstead: Required Number; halstead threshold
    • maintainability: Required Number; maintainability index threshold
  • coverage: Optional Object; configure test coverage thresholds for all projects
    • statements: Required Number; statement coverage threshold
    • branches: Required Number; branch coverage threshold
    • functions: Required Number; function coverage threshold
    • lines: Required Number; line coverage threshold
  • percentTyped: Optional Number; amount of the source code that must be typed threshold

aqua.project.json

Used to configure AQUA settings that apply to an individual project

• id: Required String; a short but unique id for all projects
• name: Required String; a descriptive name of all projects
• src: Optional Array; Where to find all source code written for all projects
• dest: Optional String; Where to output the minified version of the source code
• destName: Optional String; What file name to call the minified version of the source code
• by: Optional String; name and email of the the teams or individuals that worked on all projects
• readme: Optional String; Where to find the markdown file of all projects readme (used as the index documentation page)
• alljs: Optional Array; Where to find all the JavaScript written for all projects, including specs (used for linting)
• unit: Optional Object; Everything needed to run unit tests
  • globals: Optional Array; Where to find file(s) containing the expected global variables used by your project.
    • tests: Required Array; Where to find all of all projects unit tests
    • deps: Optional Array; Where to find the scripts all projects depends on (such as AngularJS or jQuery)
    • mocks: Optional Array; Where to find any mocks used by all projects unit tests
• e2e: Optional Object; Everything needed to run end-to-end tests
  • tests: Required Array; Where to find all of all projects end-to-end tests
  • pgobj: Optional Array; Where to find the page objects used by all projects end-to-end tests
• types: Optional Array; Where to find all type definitions used by all projects
• type: Optional String; Used to indicate the type of project (such as Node.js)

Example AQUA Config File

Below are some typical configurations that you can cut-n-paste to jump-start your AQUA setup.

Example Project Config Files

Below are some typical configurations that you can cut-n-paste to jump-start your project setup.

Web Project

Node.js Project

License

(The MIT License)

Copyright (c) 2014 Daniel Lamb [email protected]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.