clussh
v2.5.0
Published
Stream tasks to shell workers through ssh
Downloads
8
Maintainers
Readme
Clussh
Stream task execution, through ssh, to one or many host, one or many times, in parallel or in series.
Features:
- Distribute tasks upon many hosts
- Stream task from stdin
- Stream task outputs and progression stats to ldjson stream
- Support scaling, parallel execution and retry, network failure and host unavailability
Installation
npm install -g clussh
This will install three command-line cli:
- clussh - The main command line
- clussh-board - A streamable clussh progress board
- clussh-log - Pretty print clussh logs
Pre-requirement: NodeJS and its awesome universe
Usages & exemples
# Run "Hello $(hostname)" on the default worker ssh://yourusername@localhost
clussh
# Run given command on default worker
clussh --cmd 'echo "Hello world"'
clussh --cmd 'echo "Hello world"' --scale 10 # Teen times
clussh --cmd 'echo "Hello world"' --scale 10 --concurrency 2 # Two in parallel
# Run given bash script on default worker
clussh ./install.sh
# Run given bash script on to agents (with a concurrency of 2 for agent-b)
# (NB: for ssh cooking & baking reasons, the shell script must ends with an exit statement)
clussh -w ssh://[email protected] -w ssh://[email protected]?concurrency=2 ./runme.sh
# Pipe task from line delimited json stream
echo '
{ "id": "task-a", "cmd": "echo hello; sleep 1" }
{ "id": "task-b", "cmd": "echo hello; sleep 2", "scale": 10 }
{ "id": "task-c", "script": "./runme.sh", args: ["--option", "value"] }
{ "id": "task-d", "script": "./runme.sh", "worker": "ssh://[email protected]" }
' | clussh -w ssh://user@agent-a -w ssh://user@agent-b
# You can pipe infinite line delemited json stream (eg. event logs for a file wacher...)
infinite-stream | clussh
# Print human readable logs
clussh | clussh-log
# Print nice-looking progress board
clussh | clussh-board
# Print human readable logs
clussh | clussh-log
# Save logs for further exploration (and still display the board)
clussh | tee output.log | clussh-board
tail -f output.log | clussh-log # Another console...
tail -f output.log | clussh-board # Another console...
tail -f output.log | ndjson-filter 'd.type === "fail"' | prettyldjson # Another console...
Command line usage and configuration
clussh [options] [script filepath]
Options:
--help Show help [boolean]
--version Show version number [boolean]
--retry, -r How many retry for each task [number] [default: 0]
--timeout, -t Task timeout in millisecond or parseable ms time (eg. 1h,
2d or `3 days`) [string] [default: "10d"]
--worker, -w Repeatable worker uri list
[array] [default: "ssh://jponchon@localhost"]
--concurrency, -c Concurrency per worker [number] [default: 1]
--scale, -s Default scale [number] [default: 1]
--cmd Command to execute [string]
--script Script to execute (please ensure proper exit) [string]
--identity, -i List ssh identity files to use [array] [default: []]
Clussh can be configured with a .clusshrc
file (see https://www.npmjs.com/package/rc)
{
"worker": [
"ssh://[email protected]",
"ssh://[email protected]",
"ssh://[email protected]?concurrency=2"
],
"concurrency": 4,
"retry": 10
}
Above, one worker uri overide the default clussh configuration using url query string (?concurrency=2
for host agent-b.local
). This will result in two workers for agent-b.local
: one with the default concurrency (4) and another with a concurrency of 2.
Log message format
Clussh output line delemited json stream consumable by clussh-board
or clussh-log
or any other line delemited json tools (see prettyldjson for pretty print and ndjson-cli for map, reduce, filtering, etc.)
TODO: Fields documentation & messages types
API
const clussh = require('clussh')
const clusshStream = clussh({
cmd: 'hello',
worker: [ 'ssh://[email protected]', 'ssh://[email protected]' ],
retry: 3,
concurrency: 2
})
clusshStream.pipe(process.stdout)
License: MIT - Novadiscovery