npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

difftest-runner

v0.0.10

Published

A simple utility to assist in the exeuction of diff based tests.

Downloads

7

Readme

difftest-runner

difftest-runner helps run and track the output of test scripts so you can quickly determine if things are the same as they were when you decided they were 'good'.

NOTE!

I'm not a node app, I use npm for distribution because it's simple and straight-forward. I'm a collection of bash scripts, thus should run most anywhere bash does, but that's not been verified in any way. That's not to say there's no intention I run elsewhere, just that nothing's been done to make it so.

So.... What Now?

You'd be hard pressed to find something in Software Engineering that doesn't need or couldn't benefit from some automated testing. It sucks to make a trivial change to something and not be able to have an equally trivial answer to the question "Well, I wonder if that change broke anything?". So with that, testing is a must.

Testing, however can be done in a bunch of different ways. The simplest approach to writing a test is pretty straight forward:

  • do something
  • see what happens
  • decide if that's good, or make changes until it is so
  • record that output for later comparison

Using a test is even easier:

  • remember that thing you did? do it again
  • compare the output to what you decided was good (above)

This is the gist of damn near every testing framework in the world, and they all have various approaches to all aspects of that process, and generally try and be available in every single environment you can imagine. Here we try to ignore how, what, and environment and focus only on facilitating the 'given something done' what does it's output look like and, does that output look like what it used to.

All that is the long winded version of this

difftest-runner is a series of scripts that aim to make the execution of and collection of the output from a body of tests (scripts) easily repeatable such that you can compare future executions to executions deemed 'good'.

Things to Know

difftest-runner is distribted using npm. This isn't because it's javascript, or uses node because it's not and doesn't, but because npm is easy to use and good at this 'package and distribute' thing.

There is a specific directory structure that difftest-runner uses, mostly you don't have to care and can just interact with this through the commands provided but it's good to know what the hell all this is:

difftest 
  |-tests
  |-expected
  |-results
  |-filters

In the example, the root directory is called 'difftest' which is the default. While it's technically possible to change this, why would we want to have that complexity?

  • /difftest/tests - This directory contains the 'scripts' that are run to do the 'testing'. Generally I think of these items as scripts but they simply need to be exec-able, and return some deterministic output to stdout.

  • /difftest/expected - This is where the 'good' output of the scripts is stored for future comparison. Each test in the tests/ directory should have a corresponding file here which contains the output from the test that is considered good.

  • /difftest/results - This is where the output of the last run of each test is stored. Each test in the tests/ directory will create a file in this directory containing the captured output from stdout and stderr from the most recent run of the tests.

  • /difftest/filters - This contains filters to be applied to test output to make things that vary (like time stamps) fixed so comparison of output is simplified. Oh, and... If you put a filter in here named 'default' and ther is no test specific filter, that one (the default) will be used.

difftest-runner doesn't care what your tests do, a big part of this was to create something that worked the same regardless of implementation of the 'system under test'. The whole point is only that test produce output on stdout and stderr, difftest compares that to previous output.

So how do I use it?

  1. Install it

     npm install -g difftest-runner
  2. Initialize the directory you want to have tests in, I find this to be the root of my repository.

     difftest init
  3. Make a test, currently the template test is nothing more than a stub of a bash script.

     difftest create my_first_test
  4. See that the test is really there

     difftest show tests
  5. Edit the test to make it do something, this relies on the environment variable EDITOR being set.

     difftest edit my_first_test
  6. See that it fails (we haven't defined passing yet!)

     difftest run
  7. Check the results of the last test run for my_first_test

     difftest show my_first_test
  8. Tell difftest that the results of the test are good

     difftest pass my_first_test
  9. See what victory looks like!

     difftest run

Examples

Here are some examples of actual tests from somewhere else:

difftest/tests/non_existant_doc

#! /usr/bin/env bash
# vi:ft=sh
curl -s -w "\n%{http_code}" http://localhost:8080/this/key/shouldnt/exist

difftest/expected/non_existant_doc

{
  "message": "no document matching key"
}
200

difftest/tests/delete_doc

#! /usr/bin/env bash
# vi:ft=sh
KEY_PATH=`uuidgen`
curl -s http://localhost:8080/this/is/a/test/key/${KEY_PATH}
curl -s -X PUT http://localhost:8080/this/is/a/test/key/${KEY_PATH} --data '{"name":"pants"}' -H 'Content-Type: application/json'
curl -s http://localhost:8080/this/is/a/test/key/${KEY_PATH}
curl -s -X DELETE http://localhost:8080/this/is/a/test/key/${KEY_PATH}
curl -s http://localhost:8080/this/is/a/test/key/${KEY_PATH}

difftest/results/delete_doc

{
  "message": "no document matching key"
}{
  "message": "it's put"
}{"name":"pants"}{
  "message": "deleted"
}{
  "message": "no document matching key"
}

TODO

  • allow for the creation of custom test templates