Jiny - runs tests/jobs in parallel

jiny allows to run your tests/jobs/commands in parallel on the different servers. Main goal of jiny is horizontal scalability for testing.

It has 4 main features:

• upload project folder to the all servers (slaves) - jiny upload
• prepare project on the all slaves - jiny run "./prepare.sh"
• make job queue based on mask and pattern and run in parallel jiny feed *.js "mocha -R spec @"
• make job queue based on pipe and run in parallel find ... | jiny jobs

Install

• Require: tar
• npm install -g jiny

Picture

Help

> jiny 


  Usage: jiny [options] [command]

  Commands:

    master
       Starts master

    slave
       Starts slave

    upload
       Upload dir to the slaves

    run [command]
       Runs command on the all slaves. Prepare project on the all slaves. `jiny run "./prepare.sh"`

    feed [mask] [pattern]
       Find files by mask, build job based on pattern and runs jobs in parallel on different slaves. Replaces `@` -> `[file name]`. Example: `jiny --dir tests feed *.js "mocha -R spec @"`

    jobs
       Runs commands in  in parallel on the different slaves. Uses pipe: `find tests/ | grep .js | awk '{print "echo "$1" && sleep 1"}' | jiny jobs`

    stop
       Stops slaves and master

    wait.slaves [quantity]
       Waits for slaves registered in the master


  Options:

    -h, --help     output usage information
    -V, --version  output the version number
    --port [port]  master port
    --host [host]  master host
    --dir [dir]    work dir
    --detach       runs detached process: master / slave
    --live         master option to save alive slaves after exit
    --skip-upload  slave option to skip upload on this slave, if you want to use slave on the master

Local example

Make dirs

• mkdir ~/my-project # make project dir
• mkdir ~/my-project-1 # make slave dir 1
• mkdir ~/my-project-2 # make slave dir 2

Start master and slaves (in different terminals)

• jiny master --dir ~/my-project/ # start master
• jiny slave --dir ~/my-project-1/ # start slave #1
• jiny slave --dir ~/my-project-2/ # start slave #2

Prepare project (in another terminal)

> jiny run "echo prepare-project && pwd && sleep 1" # run same command on the both slaves in same time

I am client http://10.0.2.15:7500
#1 10.0.2.15 > echo prepare-project && pwd && sleep 1
prepare-project
/home/vagrant/my-project-1

#2 10.0.2.15 > echo prepare-project && pwd && sleep 1
prepare-project
/home/vagrant/my-project-2

PASSed 1s

Make fake tests

• mkdir ~/my-project/tests # make like tests dir in the project
• touch ~/my-project/tests/test1.js # make like test1
• touch ~/my-project/tests/test2.js # make like test2

Upload tests to slaves

> jiny upload # upload project to slaves

I am client http://10.0.2.15:7500
#1 10.0.2.15 uploaded dir: /home/vagrant/my-project-1/ time: 0s
#2 10.0.2.15 uploaded dir: /home/vagrant/my-project-2/ time: 0s
done 0s

#### Check uploaded data
> jiny run "pwd && ls tests" # check that tests have been uploaded

I am client http://10.0.2.15:7500
#1 10.0.2.15 > pwd && ls tests
/home/vagrant/my-project-1
test1.js
test2.js

#2 10.0.2.15 > pwd && ls tests
/home/vagrant/my-project-2
test1.js
test2.js

PASSed 0s

Find files by mask, build job based on pattern and runs jobs in parallel

• it will run commands one by one on the free slaves
• slave #1 executes test1.js, slave #2 executest test2.js in same time.
• in this example I use mocha to run these tests
• real examples looks like:
  • jiny feed --dir features *.feature "cucumber.js features/@"
  • jiny feed --dir src *Test.php "phpunit -c app/ src/@"

> jiny --dir ~/my-project/tests feed *.js "mocha -R spec tests/@"

I am client http://10.0.2.15:7500
#1 10.0.2.15 > mocha -R spec tests/test1.js


  0 passing (5ms)


#2 10.0.2.15 > mocha -R spec tests/test2.js


  0 passing (5ms)


PASSed 0s

Stop all: slaves and master

> jiny stop # stops all, slaves and master

Notes

Start master and slaves (in same terminal)

• if you want to work in one terminal use --detach option.

  • jiny master --dir ~/my-project/ --detach # start master

  • jiny slave --dir ~/my-project-1/ --detach # start slave #1

  • jiny slave --dir ~/my-project-2/ --detach # start slave #2

Find files by find and grep and pipe it to jiny jobs

• if you want to make job queue by pipe you can use this example

> find ~/my-project/tests | grep .js | awk '{print "echo "$1" && sleep 1"}' | jiny jobs

I am client http://10.0.2.15:7500

#1 192.168.1.131 > echo /home/vagrant/my-project/tests/test1.js && sleep 1
/home/vagrant/my-project/tests/test1.js

#2 192.168.1.131 > echo /home/vagrant/my-project/tests/test2.js && sleep 1
/home/vagrant/my-project/tests/test2.js

PASSed 1s

Run your tests in parallel, be scalable horizontal! Enjoy and be happy!