@niallthomson/markdown-sh
v0.3.0
Published
Its common for technical documentation to be authored using Markdown files, and for some types of content these can contain large amounts of scripts and commands to be executed by the learner. Its useful to be able to test this type of technical documenta
Downloads
19
Readme
markdown.sh
Its common for technical documentation to be authored using Markdown files, and for some types of content these can contain large amounts of scripts and commands to be executed by the learner. Its useful to be able to test this type of technical documentation to reduce the chance of regressions, breaking changes etc.
markdown.sh
can consume a set of Markdown documents, parse out the code
blocks and execute these as a suite of unit tests.
Its features include:
- Intelligently parses
bash
code blocks to extract commands that start with$
, taking in to account multi-line commands andheredoc
segments - Recursively parse all Markdown documents in a directory structure
- Order complex sets of content correctly with Frontmatter metadata (
weight
) - Additional annotations to control test behavior, such as timeouts, skipping specific scripts etc.
- A hook mechanism to plug in 'before' and 'after' actions for each test
- Ability to run subset of tests with globs (
chapter1/**
) - Support for JUnit report output format
For example, take the following Markdown:
---
title: "My Technical Docs"
weight: 10
---
# My Technical Docs
Lets execute this is a command-line:
```bash
$ ls -la
total 6
drwxr-xr-x 24 user group 768 Sep 15 14:15 .
drwxr-xr-x 143 user group 4576 Sep 9 17:20 ..
drwxr-xr-x 1 user group 8196 Aug 18 08:41 dir1
drwxr-xr-x 16 user group 512 Sep 15 18:36 dir2
drwxr-xr-x 6 user group 192 Sep 13 09:56 dir3
drwxr-xr-x 1 user group 87 Aug 26 08:59 dir4
```
Output from testing might look like:
✔ Generating test cases
✔ Building Mocha suites
Executing tests...
My Technical Docs
✔ My Technical Docs
1 passing (13ms)
success
This will execute the command ls -la
in a shell. markdown.sh
by default will figure out using the $
that this is the command to run, and understands to ignore the rest of the script block as output.
If we changed the code
block in the Markdown to this:
```bash
$ ls -la
total 6
drwxr-xr-x 24 user group 768 Sep 15 14:15 .
drwxr-xr-x 143 user group 4576 Sep 9 17:20 ..
drwxr-xr-x 1 user group 8196 Aug 18 08:41 dir1
drwxr-xr-x 16 user group 512 Sep 15 18:36 dir2
drwxr-xr-x 6 user group 192 Sep 13 09:56 dir3
drwxr-xr-x 1 user group 87 Aug 26 08:59 dir4
$ touch tester
```
Then the command ls -la
would be executed followed by touch tester
. You can also use multi-line commands:
```bash
cat syslog | \
awk ‘{print $6}’
```
And heredoc
format:
```bash
$ cat <<EOF > /tmp/yourfilehere
some file contents
EOF
```
Installing
You can get started using markdown-sh
either from NPM:
npm install -g @niallthomson/markdown-sh
markdown-sh --help
Usage: markdown-sh [options] [command]
Automated test framework for Markdown that contains shell scripts
Options:
-h, --help display help for command
Commands:
test [options] <path>
plan [options] <path>
help [command] display help for command
Usage
Basic usage:
markdown-sh test <path to content>
Where the structure of the content directory might look something like this:
├── _index.md
├── chapter1
│ ├── _index.md
│ └── introduction.md
├── chapter2
│ ├── _index.md
│ └── introduction.md
└── chapter3
├── _index.md
└── introduction.md
You can test a specific subsets of the Markdown using the --glob
parameter:
markdown-sh test --glob {chapter1,chapter3}/* .
See test-content for a concrete example.
Theres a chance markdown-sh
can run on your existing Markdown content unmodified, but it may need some help.
First, make sure that all the code blocks specify bash
as a language:
```
$ echo "This won't get run"
```
```bash
$ echo "This will get run"
```
How does it work?
The tool recursively walks the content directory looking for Markdown files, which are parsed using the unified
and remark
libraries. Metadata about the pages (title, weight) are extracted, along with any code
block that indicates bash
as the language.
This data is used to programmatically generate a set of tests using the Mocha testing framework, with each "chapter" being modelled as a separate Mocha test suite. These tests are order using the weight
metadata from Frontmatter if it exists, otherwise it is in alphabetical order.
When the test suites run the commands from the code
blocks are executed in a persistent shell session, which allows it to maintain things like environment variables set using export
for the life of the tests.
Annotations
The framework includes support for a number of annotations that can be added to the code
fence blocks in Markdown to customize the behavior for certain shell code snippets.
For example, if there are bash
segments you do not want to run you can indicate they should be skipped:
```bash test=false
$ echo "This won't get run"
```
For cases where theres a concerned a script might run for too long or not finish, you can specify a timeout in seconds (default is 60 seconds):
```bash timeout=120
$ echo "This test case will fail"
$ sleep 150
```
Here is a complete list of the available annotations:
| Annotation | Description | Default |
|-------------|--------------|---------|
| test | This script block should be executed as a test | true |
| timeout | Time limit in seconds before the script block will be marked as failed | 120 |
| hook | Name of the hook to execute for this script section | |
| hookTimeout | Time limit in seconds for the hooks to complete before the script block will be marked as failed | 300 |
| expectError | Ignore any errors that occur when the script block is executed | false |
| raw | By default a script block will be smartly interpreted to extract commands and distinguish this from sample output. Enabling this flag will executed the entire script block, assuming there is no output, and will not look for $
as a prefix for commands | false |