TE Challenge Testing Library

Contents:

• Usage
  • Typical usage
  • Using a custom compare function
  • Advanced usage
• API

Inline JavaScript code challenges use this library to assist scoring. Using this library allows code challenges on the LMS to:

• Minimize the amount of testing code included in each exercise.
• Automatically check for common error conditions (such as a missing return statement).
• Provide consist messages that are meaningful to new students.
• Have a single point of maintenance, instead of duplicating commonly used code across many challenges.

TODO: The following statement isn't true yet... The code-snippet andlocal-snippet challenges reference this library, so any JavaScript challenge testing code can reference its function.

Usage

Typical usage

To create and run challenge tests, you create an array of tests, called a suite, and then call the runSuite function. The challenge looks like this:

### !challenge
* type: code-snippet
* language: javascript
* id: e3f9a801-99b9-422a-8728-ee83792282f1
* title: Check your understanding

##### !question
In the editor, complete the code for the `calculateRectangleArea` function. 
This function calculates the area of a rectangle. It takes two input 
parameters, `length` and `width`, and returns the area which is the length
multiplied by the width.
##### !end-question

##### !placeholder
// Complete the function to calculate and return the area 
// calculateRectangleArea(1, 1) --> 1
// calculateRectangleArea(3, 2) --> 6
function calculateRectangleArea(length, width) {

}
##### !end-placeholder

This challenge presents the !question to the student, and shows a code window which contains what's in !placeholder. Then, in the !tests section:

##### !tests
__run__('calculateRectangleArea', 2);
function __run__(functionName, expectedNumberParameters) {
    // Define the array of tests    
    let tests = [
      {
          title: "calculates the area of a 1 X 1 rectangle",
          args: { length: 1, width: 1 },
          expectedValue: 1,
      },
      {
          title: "calculates the area of a 3 X 2 rectangle",
          args: { length: 3, width: 2 },
          expectedValue: 6,
      },
      {
          title: "calculates the area of a 222 X 333 rectangle",
          args: { length: 222, width: 333 },
          expectedValue: 73926,
      },
    ];
    teTestingLibrary.runSuite(functionName, _safeFindSymbol(functionName), 
        expectedNumberParameters, tests);
}

// Let JS tell if this name has been declared. Returns `undefined` if the name 
// hasn't been declared or if it's been declared but not initialized. 
// Use this for checking both functions and variables.
function _safeFindSymbol(name) {
    let theReference;
    try { theReference = eval(name); } catch (ex) { }
    return theReference;
}
##### !end-tests

Place most of the code in a __run__ function, to keep any variables out of the global namespace, potentially conflicting with code the student writes. The non-standard name reduces the chance that it'll clash with a variable the student creates. The one global line of code calls the __run__ function.

The __run__ function

The parameters to __run__ are:

• functionName: A string containing the name of the function the student is to write. The library uses this as the name of the test suite.
• expectedNumberParameters: The library verifies that the given name is a function, and that the function accepts this number of parameters. Having this information allows the library to provide students with very clear error messages.

Inside __run__, create a test suite, which is an array of tests. Each test has a descriptive title, shown in the results to the student, plus an args object containing the values to use for the arguments when calling the function. Finally, the test object includes the expected return value for the function call.

The _safeFindSymbol function

Notice that when you call runSuite, you pass a function reference. Get the function reference by first calling _safeFindSymbol.

If you pass a function reference directly, but the student misspelled the function name, they'll see a Reference error and may not know what to do. Calling _safeFindSymbol catches that exception, and the library provides clear messaging if the student didn't define the function properly.

Copy this function into the challenge verbatim. You don't need to change this function from challenge to challenge.

Top

Using a custom compare function

The library looks at the type of expectedValue and the value of the compareFunc parameter to determine how to compare the expected value to the actual test value:

• If expectedValue is not an array:
  • If compareFunc is passed in:
    • Call compareFunc, passing expected and actual values.
  • else (compareFunc is undefined)
    • Compare expected and actual using ===.
      • For string values, add an additional case-insensitive comparison to provide clearer error messages to the student.
• Else (expectedValue is an array):
  • Call function compareArraysSameMembers, which checks that the expected array and actual array match element-for-element, but not necessarily in the same sequence.
    • When comparing each element:
      • If compareFunc is passed in:
        • Call compareFunc, passing elements from the expected array and actual array.
      • else (compareFunc is undefined)
        • Compare elements from the expected and actual arrays using ===.

Usually, you want to pass a custom compareFunc when you're comparing object or arrays of objects. An example question follows:

##### !question
Write the function `firstAndLast` which accepts an array parameter and returns 
an object literal with two properties: `first` and `last` containing the 
values of the first and last array elements. 
##### !end-question

##### !placeholder
// Complete the function firstAndLast
// firstAndLast([3, 4, 5, 8]) --> {first: 3, last: 8}
// firstAndLast(['apples']) --> {first: 'apples', last: 'apples'} 
// firstAndLast([]) --> {first: undefined, last: undefined}
function firstAndLast(array) {

}
##### !end-placeholder

The tests required for this challenge are:

##### !tests
__run__('firstAndLast', 1);
function __run__(functionName, expectedNumberParameters) {
    // Define the array of tests    
    let tests = [
      {
          title: "returns an object with the first and last of 4 elements",
          args: { array: [3, 4, 5, 8]},
          expectedValue: {first: 3, last: 8},
      },      
      {
          title: "returns an object with the same first and last values for a single-element array",
          args: { array: ['apples']},
          expectedValue: {first: 'apples', last: 'apples'},
      },      
      {
          title: "returns an object with first and last = undefined for an empty array",
          args: { array: []},
          expectedValue: {first: undefined, last: undefined},
      },
    ];
    teTestingLibrary.runSuite(functionName, _safeFindSymbol(functionName), 
        expectedNumberParameters, tests, __compare__);

function __compare__(a, e) {
  return (a.first == e.first && a.last == e.last) ? '' : 
    `expected {first: ${e.first}, last: ${e.last}}, actual {first: ${a.first}, last: ${a.last}} `;
}

// Let JS tell if this name has been declared. Returns `undefined` if the name 
// hasn't been declared or if it's been declared but not initialized.
// Use this for checking both functions and variables.
function _safeFindSymbol(name) {
    let theReference;
    try { theReference = eval(name); } catch (ex) { }
    return theReference;
}
##### !end-tests

As before, the __run__ function builds an array of tests. However, the expectedValue is an object with properties first and last. The default comparison of === won't work correctly. So, you write the function __compare__, and pass that into runSuite. __compare__ checks both objects to make sure their first and last property values are equal.

The compare function must return an empty string if the values are equal, and an error message if not.

Top

Advanced usage

In the following case, the request is for the user to modify an array in-place, and then return the number of elements in the modified array. So the return value is only a portion of what the test needs to verify:

##### !question
Everyone has a list of grocery "staples", items that you want to purchase every
time you go to the grocery store. You want the user to be able to add those 
items using a single function. 
Complete the function `addStaples`, which takes an array of grocery objects 
and adds the following items to the array:
| **name** | **quantity** |
|:--------:|:------------:|
|  eggs    |       12     |
|  milk    |       1      |
`addStaples` must then return the length of the array after the 
addition of the new items.
##### !end-question

##### !placeholder
// complete the function addStaples()
// addStaples([{ name: 'apples', quantity: 6 }, 
//             { name: 'bananas', quantity: 4 }] --> 4
// addStaples([{ name: 'apples', quantity: 6 }]) --> 3
// addStaples([]) --> 2
function addStaples(groceryList) {

}
##### !end-placeholder

To accomplish this, use a custom compare function, and pass all the information you need in the expectedValue.

__run__('addStaples', 1);
function __run__(functionName, expectedNumberParameters) {
    // Define the array of tests    
    let tests = [
      {
          title: "adds staples to a grocery list with 2 items",
          args: { groceryList: [
            { name: 'apples', quantity: 6 }, 
            { name: 'bananas', quantity: 4 }]},
          expectedValue: {
            groceryList: [{ name: 'apples', quantity: 6 }, 
                          { name: 'bananas', quantity: 4 }, 
                          { name: 'eggs', quantity: 12 }, 
                          { name: 'milk', quantity: 1 }], 
            returnValue: 4},
      },
      {
          title: "adds staples to a single-item grocery list",
          args: { groceryList: [{ name: 'apples', quantity: 6 }]},
          expectedValue: {
            groceryList: [{ name: 'apples', quantity: 6 }, 
                          { name: 'eggs', quantity: 12 }, 
                          { name: 'milk', quantity: 1 }], 
            returnValue: 3},
      },
      {
          title: "adds staples to an empty grocery list",
          args: { groceryList: []},
          expectedValue: {
            groceryList: [{ name: 'eggs', quantity: 12 }, 
                          { name: 'milk', quantity: 1 }], 
            returnValue: 2},
      },
    ];

However, by default the library checks the type of value returned by the function against the type of expectedValue. Since the function should return number, but expectedValue is an object, suppress the type checking by passing false into the fifth parameter of runSuite:

teTestingLibrary.runSuite(functionName, expectedNumberParameters, 
    tests, __compareResult__, false);

Then the custom compare function can check both the return value (number), and the modified array contents:

// Compare functions return an empty string if they are equal, 
// or an error message if not
function __compareResult__(a, e, args) {
  // Called to compare the results of a test with the expected results
  let msg = '';
  // Compare the return value (actual) to the expected return value 
  if (a !== e.returnValue) {
    msg += 
    `\n*** Expected the function to return ${e.returnValue}, but it returned ${a}`;
  }

  // Compare the members of the array with what's expected
  msg += teTestingLibrary.compareArraysSameMembers(
        args.groceryList, e.groceryList, __compareItem__);

  return msg;
}

function __compareItem__(a, e) {
  // Called to compare two grocery items to determine if they are the same
  return (e.name === a.name && e.quantity === a.quantity) ? '' : 'not found';
}

The custom compare function, __compareResult__ accepts a third parameter, which is the args property of the original test object. This is the array that the student function modified.

Top

API

compareArraysSameMembers(actual, expected, compareFunc, args)

Compares two arrays to make sure they have the same membership, but in any order.

| Parameter | Type | Description | | --- | --- | --- | | actual | array | An array whose contents were actually returned by the function-under-test. | | expected | array | An array whose contents are the expected values returned by the function-under-test. | | compareFunc | function | A fn(actual, expected) which compares two elements and returns a message if they're NOT equal, '' if they are. | | args | object | The argument object from the test that's being run. It's passed through to compareFunc, because some custom compare functions need access to the args. |

This function returns a string:

• empty string if all members are equal
• if string has a value, then some members aren't equal, and the message explains the differences

comparePrimitives(actual, expected)

Simple default compare of two primitive values. Uses === for comparison.

| Parameter | Type | Description | | --- | --- | --- | | actual | any | The value actually returned by the function-under-test. | | expected | any | The expected value returned by the function-under-test. |

This function returns a string:

• An empty string if the values are equal.
• A non-empty string if the values aren't equal—the message explains the differences.

compareStrings(actual, expected)

Default compare of two string values. Uses === for comparison. Additionally checks for case-insensitive equality, and messages the user if the string differ only in casing.

| Parameter | Type | Description | | --- | --- | --- | | actual | string | The value actually returned by the function-under-test. | | expected | string | The expected value returned by the function-under-test. |

This function returns a string:

• An empty string if the values are equal.
• A non-empty string if the values aren't equal—the message explains the differences.

runSuite(functionName, functionRef, expectedNumberParameters, tests, compareFunc, checkReturnType, beforeEachFunc)

Run every test in a suite (an array of tests).

| Parameter | Type | Description | | --- | --- | --- | | functionName | string | The name is the function expected in the student code. | | functionRef | reference | Reference to the function-under-test, as returned by _safeFindSymbol (see Comments). | | expectedNumberParameters | number | Number of params the function-under-test must declare | | tests | Test[] | array of test objects, which have properties: title: description or title of text (in the "it"), args: arguments to the function under test, expectedValue: expected return value. | | compareFunc | function | function that compares actual to expected to override default compare | | checkReturnType | boolean | Whether to check the type returned by the function (default: true) | | beforeEachFunc | function | Function to call before every test (it) |

Comments

To use this function: In your local code, include this function:

// Let JS tell if this name has been declared. Returns `undefined` if the name
// hasn't been declared or if it's been declared but not initialized.
function _safeFindSymbol(name) {
    let theReference;
    try { theReference = eval(name); } catch (ex) { }
    return theReference;
}

Then call verifyFunction like this:

teTestingLibrary.runSuite(functionName, _safeFindSymbol(functionName), 
    expectedNumberParameters, tests);

runTest(funcUnderTest, title, args, expected, compareFunc, checkReturnType)

Runs a single test. Called by runTests—you don't call this function directly.

| Parameter | Type | Description | | --- | --- | --- | | funcUnderTest | reference | Reference to the function-under-test. | | title | string | title property of the test object. Used as the description in the it. | | args | string | args property of the test object. Used to call the function-under-test. | | expected | string | expected property of the test object. Used to verify the actual result. | | compareFunc | function | function that compares actual to expected to override default compare | | checkReturnType | boolean | Whether to check the type returned by the function (default: true) |

runTests(funcUnderTest, tests, compareFunc, checkReturnType)

Runs each test in an array of tests. Called by runSuite—you don't call this function directly.

| Parameter | Type | Description | | --- | --- | --- | | funcUnderTest | reference | Reference to the function-under-test. | | tests | Test[] | array of test objects | | compareFunc | function | function that compares actual to expected to override default compare | | checkReturnType | boolean | Whether to check the type returned by the function (default: true) |

verifyFunction(functionName, theFunction, expectedNumberParameters)

Given a variable reference, check it's type to make sure it's a function and check the number of parameters are what's expected.

| Parameter | Type | Description | | --- | --- | --- | | functionName | string | The name is the function expected in the student code. | | theFunction | reference | A reference returned by _safeFindSymbol (see Comments). | | expectedNumberParameters | number | The number of parameters the function should accept. |

Comments

To use this function: In your local code, include this function:

// Let JS tell if this name has been declared. Returns `undefined` if the name
// hasn't been declared or if it's been declared but not initialized.
function _safeFindSymbol(name) {
    let theReference;
    try { theReference = eval(name); } catch (ex) { }
    return theReference;
}

Then call verifyFunction like this:

verifyFunction(functionName, _safeFindSymbol(functionName), 
    expectedNumberParameters);

verifyVariable(variableName, theVariable, expectedType)

Given a variable reference, check it's type against what's expected, and provide an error message if an unexpected type is encountered.

| Parameter | Type | Description | | --- | --- | --- | | variableName | string | The name is the variable expected in the student code. | | theVariable | reference | A reference returned by _safeFindSymbol (see Comments). | | expectedType | string | The JavaScript type ('string', 'number') the variable should be. If the variable should be an array, use 'array'. |

Comments

To use this function: In your local code, include this function:

// Let JS tell if this name has been declared. Returns `undefined` if the name
// hasn't been declared or if it's been declared but not initialized.
function _safeFindSymbol(name) {
    let theReference;
    try { theReference = eval(name); } catch (ex) { }
    return theReference;
}

Then call verifyVariable like this:

verifyVariable(variableName, _safeFindSymbol(functionName), expectedType);