activitypub-testing

tools for testing implementations of ActivityPub, a decentralized social networking protocol.

How to use the activitypub-testing CLI

Prerequisites

• have node and npm on your PATH. If you don't have them, install node.js.

Ad-hoc Usage

If you have npx installed (which comes with npm), you can use the activitypub-testing cli without downloading the source code or installing the package globally.

This can be useful to quickly use the tool, but it also can be a bit slow, so see below for how to install the activitypub-testing CLI for direct usage.

Throughout this README, ⚡ is used in code blocks to indicate a shell prompt. Yours might be $. Copy the rest of the command other than the shell prompt to follow along.

⚡ npx activitypub-testing help

activitypub-testing

What?
  activitypub-testing is a cli for testing implementations of ActivityPub[0]

Usage:
  # browse all the tests
  activitypub-testing get tests [--output=<mediaType>]

  # inspect a single test
  activitypub-testing get test (--uuid=<uuid>|--slug=<slug>) [--output=<mediaType>]

  # run a test
  activitypub-testing run test (--uuid=<uuid>|--slug=<slug>) --input.anyInputName=<input.anyInputName>

  # run tests on a specific ActivityPub Actor
  activitypub-testing test actor <actor-uri> [--output=<mediaType>]

  # print this help
  activitypub-testing [--help|-h] [help]

Options:
  -h --help        Show this help text.
  -o --output      Choose output media type (default `text`, also allows `json`, `yaml`)
  --slug           slug (aka URL-path-friendly human-readable name) of selection
  --uuid           UUID (rfc4122) identifier of selection

Examples:
  # runs test inbox-must-be-an-orderedcollection[1] against a valid actor fetched via https
  activitypub-testing \
    test \
    --uuid '5e94d155-ed4a-4d71-b797-d7c387736ecf' \
    --input.object="$(curl -s 'https://socialweb.coop/activitypub/actors/with-empty-inbox.json')"

  # run tests on a specific ActivityPub Actor (pipe to `jq` to pretty print JSON)
  activitypub-testing \
    test actor \
    https://socialweb.coop

[0]: https://en.wikipedia.org/wiki/ActivityPub
[1]: https://socialweb.coop/activitypub/test-cases/inbox-must-be-an-orderedcollection/

Installing activitypub-testing on your PATH

You can install activitypub-testing for usage without having to type npx:

⚡ npm install -g activitypub-testing

npm install -g installs activitypub-testing to your shell PATH. Then the following should work in your command prompt:

⚡ activitypub-testing help

Test an ActivityPub Actor

⚡ activitypub-testing test actor https://socialweb.coop

As tests are run, newline-delimited JSON (aka ndjson) descriptions of the results are streamed to stdout.

That means you can compose activitypub-testing with other tools in a pipeline.

e.g. head can truncate output:

⚡ activitypub-testing test actor https://socialweb.coop | head -n1
{"type":["Assertion"],"result":{"outcome":"passed"},"test":{"slug":"following-collection-must-be-a-collection","url":"https://socialweb.coop/activitypub/test-cases/following-collection-must-be-a-collection","description":"tests whether an ActivityPub Object has a `following`` collection with an appropriate Collection type","name":"An ActivityPub Actor Object's `following` Collection Must be a Collection","uuid":"018c3e17-a1bd-7040-8007-4cd3b9063288"},"input":{"object":"{\n  \"type\": [\n    \"Organization\"\n  ],\n  \"inbox\": \"https://socialweb.coop/inbox\",\n  \"outbox\": \"https://socialweb.coop/outbox\",\n  \"followers\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"following\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"liked\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"likes\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"shares\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"id\": \"https://socialweb.coop/\",\n  \"@context\": [\n    \"https://www.w3.org/ns/activitystreams\"\n  ]\n}"},"@context":["https://www.w3.org/ns/activitystreams","https://socialweb.coop/ns/testing/context.json"]}
stdout closed w/ EPIPE

Processing Results with jq

jq is very useful for processing JSON, and it's a recommended companion to activitypub-testing.

⚡ activitypub-testing test actor https://socialweb.coop | jq 
{
  "type": [
    "Assertion"
  ],
  "result": {
    "outcome": "passed"
  },
  "test": {
    "slug": "following-collection-must-be-a-collection",
    "url": "https://socialweb.coop/activitypub/test-cases/following-collection-must-be-a-collection",
    "description": "tests whether an ActivityPub Object has a `following`` collection with an appropriate Collection type",
    "name": "An ActivityPub Actor Object's `following` Collection Must be a Collection",
    "uuid": "018c3e17-a1bd-7040-8007-4cd3b9063288"
  },
  "input": {
    "object": "{\n  \"type\": [\n    \"Organization\"\n  ],\n  \"inbox\": \"https://socialweb.coop/inbox\",\n  \"outbox\": \"https://socialweb.coop/outbox\",\n  \"followers\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"following\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"liked\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"likes\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"shares\": {\n    \"type\": \"OrderedCollection\"\n  },\n  \"id\": \"https://socialweb.coop/\",\n  \"@context\": [\n    \"https://www.w3.org/ns/activitystreams\"\n  ]\n}"
  },
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://socialweb.coop/ns/testing/context.json"
  ]
}
# more JSON objects omitted for README brevity

If you want to slurp all the output objects into a single JSON array, you can do that with jq's --slurp aka -s flag.

⚡ activitypub-testing test actor https://socialweb.coop | jq -s
[
  {
    "type": [
      "Assertion"
# more JSON omitted for README brevity

Get Tests Collection

activitypub-testing get tests will get a collection of available tests. The default behavior is to format it to be readable by a human. Pass -o json for json output.

⚡ activitypub-testing get tests
name: ActivityPub Tests
type:
  - Collection
items:
  - slug: actor-objects-must-have-inbox-outbox-properties
    uuid: acaacb5f-8f7e-4f28-8d81-c7955070a767
    url: https://socialweb.coop/activitypub/test-cases/acaacb5f-8f7e-4f28-8d81-c7955070a767
    id: urn:uuid:acaacb5f-8f7e-4f28-8d81-c7955070a767
  - slug: actor-must-serve-as2-object-to-get
    uuid: e7ee491d-88d7-4e67-80c8-f74781bb247c
    url: https://socialweb.coop/activitypub/test-cases/e7ee491d-88d7-4e67-80c8-f74781bb247c
    id: urn:uuid:e7ee491d-88d7-4e67-80c8-f74781bb247c
  - slug: inbox-must-be-an-orderedcollection
    uuid: 5e94d155-ed4a-4d71-b797-d7c387736ecf
    url: https://socialweb.coop/activitypub/test-cases/5e94d155-ed4a-4d71-b797-d7c387736ecf
    id: urn:uuid:5e94d155-ed4a-4d71-b797-d7c387736ecf
  - slug: outbox-must-be-an-orderedcollection
    uuid: 4af549f4-3797-4d99-a151-67c3d8feaa46
    url: https://socialweb.coop/activitypub/test-cases/4af549f4-3797-4d99-a151-67c3d8feaa46
    id: urn:uuid:4af549f4-3797-4d99-a151-67c3d8feaa46
  - slug: shares-collection-must-be-a-collection
    uuid: b03a5245-1072-426d-91b3-a3d412d45ae8
    url: https://socialweb.coop/activitypub/test-cases/b03a5245-1072-426d-91b3-a3d412d45ae8
    id: urn:uuid:b03a5245-1072-426d-91b3-a3d412d45ae8
  - slug: likes-collection-must-be-a-collection
    uuid: 200b9bc8-aae3-46f2-a6ab-5366042c0f6e
    url: https://socialweb.coop/activitypub/test-cases/200b9bc8-aae3-46f2-a6ab-5366042c0f6e
    id: urn:uuid:200b9bc8-aae3-46f2-a6ab-5366042c0f6e
  - slug: liked-collection-must-be-a-collection
    uuid: 018c3df2-d6d8-7f62-805b-b71a96cc6170
    url: https://socialweb.coop/activitypub/test-cases/018c3df2-d6d8-7f62-805b-b71a96cc6170
    id: urn:uuid:018c3df2-d6d8-7f62-805b-b71a96cc6170
  - slug: followers-collection-must-be-a-collection
    uuid: 018c3e08-611f-7e56-9f45-2fe5e4877d4e
    url: https://socialweb.coop/activitypub/test-cases/018c3e08-611f-7e56-9f45-2fe5e4877d4e
    id: urn:uuid:018c3e08-611f-7e56-9f45-2fe5e4877d4e
  - slug: following-collection-must-be-a-collection
    uuid: 018c3e17-a1bd-7040-8007-4cd3b9063288
    url: https://socialweb.coop/activitypub/test-cases/018c3e17-a1bd-7040-8007-4cd3b9063288
    id: urn:uuid:018c3e17-a1bd-7040-8007-4cd3b9063288
  - slug: outbox-post-servers-must-return-a-201-created-http-code
    uuid: 723afcbb-118d-433e-8ab4-560ffca93582
    url: https://socialweb.coop/activitypub/test-cases/723afcbb-118d-433e-8ab4-560ffca93582
    id: urn:uuid:723afcbb-118d-433e-8ab4-560ffca93582
"@context":
  - https://www.w3.org/ns/activitystreams

Get a single Test

Get a Test by Slug

Every test has a human-readable slug, which can be useful for identifying and selecting specific tests.

Get all info about a specific teest

⚡ activitypub-testing get test --slug actor-must-serve-as2-object-to-get
type:
  - TestCase
description: This rule checks that URLs of ActivityPub objects can be resolved
  to a representation with well-known media type for further processing.
failedCases:
  - name: nginx 404 response body
    inputs:
      id: https://bengo.is/404
      time: T1M
    result:
# lots more yaml omitted

See here for full example output from the above command.

Every test has a human-readable description of the test in markdown as a markdown property

⚡ activitypub-testing get test \
    --slug actor-must-serve-as2-object-to-get \
    -o json \
  | jq -r .markdown

Get a Test by UUID

⚡ activitypub-testing get test --uuid e7ee491d-88d7-4e67-80c8-f74781bb247c

Get Test Input Description

⚡ activitypub-testing get test \
    --slug actor-must-serve-as2-object-to-get \
    -o json \
  | jq -r .inputs  
{
  "id": {
    "help": "identifier of an ActivityPub Object hosted at an ActivityPub Server",
    "type": "xsd:anyUri",
    "rangeIncludes": [
      "https://www.w3.org/ns/activitystreams#Actor"
    ],
    "required": true
  },
  "authorization": {
    "help": "proof of authorization to retrieve the object identified by input `id`"
  },
  "time": {
    "help": "amount of time allowed to run test. This is meant to configure the limit for how long this test will wait for network requests. MUST be an [RFC3339 `dur-time`](https://datatracker.ietf.org/doc/html/rfc3339#appendix-A)",
    "required": true,
    "type": [
      "rfc3339:dur-time",
      "TimeLimit"
    ]
  }
}

Run a Single Test

The run test command takes a test selector and --input.{inputName}={inputValue} flags that get parsed to build the test input, which is then provided to the test to run. See Get Test Input Description for how to describe the input for a test.

run test by slug

In this example, the time and id inputs are defined by the test selected by the --slug flag.

⚡ activitypub-testing run test \
  --slug actor-must-serve-as2-object-to-get \
  --input.time="T1M" \
  --input.id="https://bengo.is/actor.json"
{
  "type": "Assertion",
  "test": {
    "id": "urn:uuid:e7ee491d-88d7-4e67-80c8-f74781bb247c",
    "uuid": "e7ee491d-88d7-4e67-80c8-f74781bb247c",
    "url": "https://socialweb.coop/activitypub/test-cases/actor-must-serve-as2-object-to-get/",
    "slug": "actor-must-serve-as2-object-to-get"
  },
  "input": {
    "time": "T1M",
    "id": "https://bengo.is/actor.json"
  },
  "result": {
    "outcome": "passed"
  },
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://socialweb.coop/ns/testing/context.json"
  ]
}

run test by URL

Note: activitypub-testing tests run by URL MUST be resolvable to a Test Module as specified in FEP-c551: Use ECMAScript Modules to Create Conformance Tests for Fediverse Enhancement Proposals

In this example, the test is selected by URL with --url=<url> with <url> to JavaScript file in a activitypub-testing-fep-521a package as an example of how anyone can publish Test Modules for FEP Tests.

⚡ activitypub-testing run test \
  --url=https://codeberg.org/socialweb.coop/activitypub-testing-fep-521a/raw/branch/main/fep/521a/actor-objects-must-express-signing-key-as-assertionMethod-multikey.js \
  --input.actor='{"type":"Person","assertionMethod":[{"type":"Multikey","id": "https://example.com/#ed25519-key","controller": "https://example.com/","publicKeyMultibase": "6MkrJVnaZkeFzdQyMZu1cgjg7k1pZZ6pvBQ7XJPt4swbTQ2"}]}'

Developing this Repository

Directories

• dist - built from src, e.g. typescript .d.ts files
• doc - documentation
• etc - configuration files
• issues - known issues. Feel free to add an issue here
• src - source code for activitypub-testing

Some directory names are loosely inspired by linux equivalents.

Developing ActivityPub Test Cases

Each test case has a directory in ./src/activitypub-tests/ named by its slug. In each test case directory, there is:

• {slug}.md - human readable version of the test case specification
• {slug}.js - JavaScript module implementation of the test case specification
• {slug}.test.js - JavaScript tests for the test case implementation for use with node.js test runner. These get run by node --test

Developing the activitypub-testing CLI

See ./src/cli.js for the source code.

You should be able to run the cli.js script like

⚡ ./src/cli.js --uuid=test-uuid
no test found with uuid of test-uuid

npm scripts

These are in ./package.json and each has a name. Run like npm run <name>

• start - runs the activitypub-testing cli
• build - build the src code into dist
• lint - check source code against lint rules (js and markdown)
• test - run unit tests
• test:coverage - run unit tests and show code test coverage stats

Contributing to this project

See the separate CONTRIBUTING.md for details on our contributer license agreement requested of all contributors.