npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@mojojs/user-agent

v0.11.4

Published

A powerful user-agent for Node.js and browsers

Downloads

74

Readme

Coverage Status npm

A powerful user-agent for Node.js and browsers. Written in TypeScript. VERY EXPERIMENTAL!

import UserAgent from '@mojojs/user-agent';

const ua = new UserAgent();
const res = await ua.get('https://mojolicious.org');
const dom = await res.html();
const title = dom.at('title').text();

The API is heavily inspired by the Fetch Standard and should feel familar if you've used fetch before.

User-Agent Options

The user-agent can be initialized with a few options, but none of them are required.

const ua = new UserAgent({

  // Base URL to be used to resolve all relative request URLs with
  baseURL: 'http://127.0.0.1:3000',

  // Disable TLS certificate validation (only Node.js)
  insecure: true,

  // Keep-alive timeout, disabled with `null`, defaults to 1000 (only Node.js)
  keepAlive: 3000,

  // Maximum number of redirects to follow, defaults to 20 (only Node.js)
  maxRedirects: 5,

  // Name of user-agent to send with `User-Agent` header (only Node.js)
  name: 'mojoUA/1.0'
});

Request Config

Every request is represented by a config object that contains various properties to describe every part of the HTTP request.

const res = await ua.request({

  // HTTP method for request
  method: 'GET',

  // URL of request target as a string or URL object, may be be relative to `ua.baseURL`
  url: new URL('https://mojolicious.org'),

  // Headers to include in request
  headers: {Accept: '*/*', Authorization: 'token 123456789abcdef'},

  // Object with key/value pairs to be sent with the query string
  query: {fieldA: 'first value', fieldB: 'second value'},

  // Request body as a string
  body: 'Some content to send with request',

  // Data structure to be send in JSON format
  json: {hello: ['world']},

  // Object with key/value pairs to be sent in `application/x-www-form-urlencoded` format
  form: {fieldA: 'first value', fieldB: 'second value'},

  // Object with key/value pairs or `FormData` object to be sent in `multipart/form-data` format
  formData: {fieldA: 'first value', fieldB: 'second value', fieldC: {content: 'third value', filename: 'foo.txt'}},

  // Basic authentication
  auth: 'user:password',

  // An `AbortSignal` to abort the request
  signal: controller.signal
});

The request method returns a Promise that resolves with a response object, right after the response status line and headers have been received. But before any data from the response body has been read, which can be handled in a separate step later on.

Request Shortcuts

Since every request includes at least method and url values, there are HTTP method specific shortcuts you can use instead of request.

const res = await ua.delete('https://mojolicious.org');
const res = await ua.get('https://mojolicious.org');
const res = await ua.head('https://mojolicious.org');
const res = await ua.options('https://mojolicious.org');
const res = await ua.patch('https://mojolicious.org');
const res = await ua.post('https://mojolicious.org');
const res = await ua.put('https://mojolicious.org');

All remaining config values can be passed with a second argument to any one of the shortcut methods.

const res = await ua.post('/search', {form: {q: 'mojo'}});

Response Headers

Status line information and response headers are available right away with the response object.

// Status code and message
const statusCode = res.statusCode;
const statusMessage = res.statusMessage;

// Headers
const contentType = res.get('Content-Type');

// 2xx
const isSuccess = res.isSuccess;

// 3xx
const isRedirect = res.isRedirect;

// 4xx
const isClientError = res.isClientError;

// 5xx
const isServerError = res.isServerError;

// 4xx or 5xx
const isError = res.isError;

Response Body

The reponse body can be received in various formats. Most of them will result once again in a new Promise, resolving to different results however.

// ReadableStream
const stream = res.body;

// String
const text = await res.text();

// Uint8Array
const data = await res.data();

// Parsed JSON
const data = await res.json();

// Parsed HTML via `@mojojs/dom`
const dom = await res.html();
const title = dom.at('title').text();

// Parsed XML via `@mojojs/dom`
const dom = await res.xml();

// `stream.Readable` (only Node.js)
const stream = res.createReadStream();

// Pipe content to `stream.Writable` object (only Node.js)
await res.pipe(process.stdout);

For HTML and XML parsing @mojojs/dom will be used. Making it very easy to extract information from documents with just a CSS selector and almost no code at all.

Cookies

By default, for Node.js, a tough-cookie based cookie jar will be used for state keeping, and you can reconfigure it however you like.

ua.httpTransport.cookieJar.allowSpecialUseDomain = true;

Of course you can also just disable cookies completely.

ua.httpTransport.cookieJar = null;

In browsers the native browser cookie jar will be used instead.

Hooks

Hooks can be used to extend the user-agent and run code for every HTTP request.

// Add a header to every HTTP request
ua.addHook('request', async (ua, config) => {
  config.headers['X-Bender'] = 'Bite my shiny metal ass!';
});

// Add a query parameter to every HTTP request
ua.addHook('request', async (ua, config) => {
  config.url.searchParams.append('hello', 'mojo');
});

Timeouts

You can use an AbortController to make sure a request does not take longer than a certain amount of time. Once aborted the promise returned by ua.get() will reject.

const controller = new AbortController();
const signal = controller.signal;
setTimeout(() => controller.abort(), 3000);
const res = await ua.get('https://mojojs.org', {signal});

Compression

Responses with gzip or deflate content encoding will be decompressed transparently.

Introspection

With Node.js you can set the MOJO_CLIENT_DEBUG environment variable to get some advanced diagnostics information printed to STDERR.

$ MOJO_CLIENT_DEBUG=1 node myapp.js
-- Client >>> Server
GET /hello.html
accept: */*
accept-language: *
sec-fetch-mode: cors
accept-encoding: gzip, deflate

-- Client <<< Server
200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 12
Date: Mon, 02 May 2022 23:32:34 GMT
Connection: close

Hello World!

Installation

All you need is Node.js 18 (or newer).

$ npm install @mojojs/user-agent

Support

If you have any questions the documentation might not yet answer, don't hesitate to ask in the Forum, on Matrix, or IRC.