nyuu
v0.4.2
Published
Flexible usenet binary poster
Downloads
118
Maintainers
Readme
Nyuu
A powerful beast hidden by a deceptively cute exterior
Nyuu is a command-line binary usenet poster.
Nyuu is designed primarily to be fast/efficient, reliable and customizable, exposing all the interesting bits and pieces. From what I’ve seen, Nyuu excels more than any other usenet posting client in these aspects.
Nyuu runs on top of node.js (which everyone knows is bad ass rock star tech) with minimal dependencies. It does require one compiled module, yencode, for ultra fast yEnc and CRC32 calculation.
Features
All the standard stuff: i.e. Nyuu doesn’t miss out on what you need
NNTP SSL support
yEnc encoding
NZB creation, with minification & compression
Sub-directory handling
Fast & efficient: Nyuu is amongst the fastest (if not the fastest) usenet posters, designed to run on low power, overloaded servers with >= 1Gbps connections
Multiple uploading connections
Post checking is asynchronous (minimal impact on speed)
Fastest yEnc and CRC32 implementations
Buffered async I/O (minimise the effects of slow disks)
No temporary files, no disk writes (except for writing the NZB output)
No unnecessary disk reads - only performs a single, sequential read pass over files
If a post needs to be resubmitted, due to a check failure, and it is not cached, it will need to be re-read off disk
PAR2 generation (when integrated) will require an extra initial pass of just the first 16KB of each file, and may require multiple read passes if all recovery blocks cannot fit in memory
Reliable: Nyuu was designed for automation and its reliability requirements
Requests are all retryable
Able to recover from connection failures/dropouts
Timeouts and limits to deal with unexpected server hangs or faults
Can selectively skip/ignore some errors
Post checking (aka header checks), with multiple attempts and post retrying
Unusual or unexpected events are logged
Includes some optional workarounds for server bugs
Highly configurable: tuning knobs for everything
Lots of connection and SSL options
NNTP article posting: article/line sizes, header customisation
Configure timeouts, delays and error handling parameters
NZB meta tags
Unique features: the not so usual stuff
Pipe input/output from/to processes instead of files, plus the ability to pipe out a copy of read input to an external process without incurring additional disk reads
Extensive upload diagnostic details available (via optional TCP/HTTP status server) to help tune settings or find problems
Installation & Requirements
Pre-Built Binaries
Static pre-built binaries with Node 4.x may be found on the releases page if I can be bothered to provide them. These can simply be extracted and run.
Arch
Archlinux users can install the latest binary-release by installing nyuu-bin
from the AUR.
Please note that this Package is maintained by a third party and not officially supported. If you have an issue please contact the maintainer.
Install Via NPM
If NPM is installed (usually comes bundled with node.js), the following command can be used to install Nyuu:
npm install -g nyuu --production
You’ll then be able to run Nyuu via the nyuu command.
If the npm command isn’t available, it can probably be installed via your
package manager (apt-get install npm
for Debian), or see the following section
titled “Node.js” for more details.
If you get a gyp ERR! stack Error: EACCES: permission denied
error when installing, try the following command instead:
npm install -g nyuu --production --unsafe-perm
Note: on some systems, you won’t get an error during installation, but when running Nyuu, it crashes with the error Error: Cannot find module './build/Release/yencode.node'
. If that is the case, you may need to run the above command to get it to install properly.
You can then later uninstall Nyuu via:
npm uninstall -g nyuu
Install From Source
Note that code from Git is considered to be unstable (or rather, less stable than release versions). Stable packages can be found on the releases page.
Node.js
Nyuu should run on node.js 0.10.x and later. Recent Linux distributions should have nodejs in their repositories, otherwise, see installing via package manager or NodeSource. For Windows/OSX builds, they can be found here. Although node.js 0.10.x is supported, newer versions of Node (>=4 recommended) provide greatly improved SSL performance.
Dependencies
If you have NPM installed (may come with your install of NodeJS, or you may need to install it like you did with NodeJS if your package system doesn’t include them together), the following command (executed inside the Nyuu directory) is all you need to set up the dependencies:
npm install --production
If you don’t want to use NPM, you can use your package manager instead if it has all the necessary packages - on Debian 8 / Ubuntu 14.04:
apt-get install node-async
Alternatively, you can grab the dependencies manually: create a directory named node_modules in the Nyuu directory. Inside of which, place async.
For yencode, you’ll need to place it in the node_modules directory as well, then follow the installation instructions.
At the end, the folder structure should resemble something like (not all files shown):
Nyuu/
- bin/
- lib/
- node_modules/
- async/
- package.json
- yencode/
- index.js
package.json
“no suitable image found” error on MacOS 10.15
Due to security changes in OSX 10.15, libraries may require code signing to work. To deal with this, you’ll either need to disable this security option or codesign the built yencode module. Note that I do not have OSX and can’t provide much support for the platform.
Executable
Once dependencies have been installed, Nyuu can be run via node bin/nyuu
or
nodejs bin/nyuu
. If you wish to just use nyuu
instead, you need to link to
it from somewhere your PATH environment points to. For example, on Linux you
might do
ln -s "`pwd`/bin/nyuu.js" /usr/bin/nyuu
chmod a+x bin/nyuu.js
(if Node is running via nodejs instead of node, you can edit the first line in bin/nyuu.js to fix this)
For Windows, you can make a file named nyuu.cmd and place it in your system directory, with the following contents:
@"C:\node\node.exe" "C:\path\to\nyuu\bin\nyuu.js" %*
(obviously, fix the paths to what they actually are)
Development
Running Tests
Tests are run via mocha, installable via npm install -g mocha
, and can be
run simply by using the mocha
command inside Nyuu’s root directory.
Note that some test cases test functionality of timeouts; to reduce time it
takes to run these tests, timeouts are set relatively small, which means that a
slow computer may not be able to service them as expected.
Building Binary
A basic script to compile the Nyuu binary is provided in the nexe folder. The script has been tested with NodeJS 12.20.0 and may work on other 12.x.x versions. For NodeJS 8.x.x and older, see section below.
- If you haven’t done so already, do an
npm install
in Nyuu’s folder to ensure its dependencies are available - Enter the nexe folder and do an
npm install
to pull down required build packages (note, nexe requires NodeJS 10 or greater) - If desired, edit the variables at the top of nexe/build.js
- Run
node build
. If everything worked, there’ll eventually be a nyuu or nyuu.exe binary built. If it fails during compilation, enter the nexe/build/12.20.0 (or whatever version of NodeJS you’re using) and get more info by:- Linux: build using the
make
command - Windows: build using
vcbuild.bat
followed by build options, e.g.vcbuild nosign x86 noetw intl-none release static no-cctest without-intl ltcg
- Linux: build using the
- If building for Linux, optionally
strip
the executable to reduce size
Building for NodeJS 8.x.x or older
Compiling Nyuu into a single binary can be done via nexe 1.x. There is a little complication with bundling the yencode module, but a rather fragile script has been supplied in nexe1/build.js to help with the process. The following general steps need to be taken:
- Ensure that nexe is installed (doesn’t need to be globally installed) and its requirements met
- Download a Node.js source package. The script has mostly been tested with Node 4.x.x, it may work with other versions
- The required Nyuu libraries need to be installed into the node_modules folder
- Inside the nexe1 folder (the one containing build.js), create the following two folders: node and yencode-src
- Inside the node folder, create a folder with the version number of the package you downloaded in step 2, for example “4.9.1”. Inside this folder, create one named “_” and place the downloaded sources in this folder. After doing this, the file nexe1/node/x.x.x/_/node.gyp should exist, where x.x.x is the node version number
- Inside the yencode-src folder, copy the source code for the yencode (v1.0.x) module
- Edit nexe1/build.js; options that are likely to be edited are at the top of the file. You’ll likely need to change nodeVer to be the version of node you’re using
- You may need to edit bin/nyuu.js to fix the location of nexe’s package.json path (search bin/nyuu.js for
nexe/package.json
and update if necessary) - In the nexe1 folder, run build.js. This script patches node to embed the yencode module, and customises a few compiler options, then calls nexe to build the final executable. If it worked, you should get a binary named nyuu or nyuu.exe in the nexe1 folder
Note that this will be built with the -flto
option on non-Windows platforms.
If this causes build failures, your system’s ar
utility may not support LTO
objects, which can be worked around if you have gcc-ar
installed by issuing a
command like export AR=gcc-ar
Usage
For command line usage, see here, or here for a summarized list of options.
Default Configuration
As entering long command lines may be tiresome, you may use a config file with
Nyuu via the --config
or -C
option. You may use the config-sample.json
file as reference to what a config file should be like.
If a config file isn't specified, Nyuu will also search the NYUU_CONFIG
environment variable for a default configuration (saving you from needing to
specify this file on every run, if you set the environment up in your shell).
Planned Features
Integrate ParPar for streaming PAR2 creation
Streaming 7-Zip creation
Improve multi-server support for both posting and checking
Repost missing articles from NZB, and/or some sort of resumption support
SOCKS proxy support
A web (HTTP) interface would be nice as an alternative to the command line interface; not sure if it will be done however
Not planned
- RAR support, since 7-Zip should cover everything
Alternatives
A list of Usenet posters I’ve come across can be found here.
Here's a benchmark comparison between a few of the command line posters.
License
Nyuu is Public Domain or CC0 (or equivalent) if PD isn’t recognised. Use her as you will, at your own risk of course (don’t come back crying if you lose a limb or two).