npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

tldjs

v2.3.1

Published

JavaScript API to work against complex domain names, subdomains and URIs.

Downloads

340,713

Readme

tld.js Backers on Open Collective Sponsors on Open Collective Build Status

tld.js is a Node.js module written in JavaScript to work against complex domain names, subdomains and well-known TLDs.

It answers with accuracy to questions like what is mail.google.com's domain?, what is a.b.ide.kyoto.jp's subdomain? and is https://big.data's TLD a well-known one?.

tld.js runs fast, is fully tested and is safe to use in the browser (with browserify, webpack and others). Because it relies on Mozilla's public suffix list, now is a good time to say thank you Mozilla!

Install

# Regular install
npm install --save tldjs

# You can update the list of well-known TLD during the install
npm install --save tldjs --tldjs-update-rules

The latter is useful if you significantly rely on an up-to-date list of TLDs. You can list the recent changes (changes Atom Feed) to get a better idea of what is going on in the Public Suffix world.

Using It

const {parse, tldExists} = require('tldjs');

// Checking only if TLD exists in URL or hostname
// First TLD exists; the second does not.
console.log(tldExists('https://www.bbc'));
console.log(tldExists('tld.unknown'));

// Retrieving hostname related informations of a given URL
parse('http://www.writethedocs.org/conf/eu/2017/');

👋 Try it your browser to see how it works. ⬇️ Read the documentation below to find out the available functions.

tldjs.parse()

This methods returns handy properties about a URL or a hostname.

const tldjs = require('tldjs');

tldjs.parse('https://spark-public.s3.amazonaws.com/dataanalysis/loansData.csv');
// { hostname: 'spark-public.s3.amazonaws.com',
//   isValid: true,
//   isIp: false,
//   tldExists: true,
//   publicSuffix: 's3.amazonaws.com',
//   domain: 'spark-public.s3.amazonaws.com',
//   subdomain: ''
// }

tldjs.parse('gopher://domain.unknown/');
// { hostname: 'domain.unknown',
//   isValid: true,
//   isIp: false,
//   tldExists: false,
//   publicSuffix: 'unknown',
//   domain: 'domain.unknown',
//   subdomain: ''
// }

tldjs.parse('https://192.168.0.0')
// { hostname: '192.168.0.0',
//   isValid: true,
//   isIp: true,
//   tldExists: false,
//   publicSuffix: null,
//   domain: null,
//   subdomain: null
// }

| Property Name | Type | | | --- | --- | --- | | hostname | String | | | isValid | Boolean | Is the hostname valid according to the RFC? | | tldExists | Boolean | Is the TLD well-known or not? | | publicSuffix| String | | | domain | String | | | subdomain | String | |

Single purpose methods

These methods are shorthands if you want to retrieve only a single value.

tldExists()

Checks if the TLD is well-known for a given hostname — parseable with require('url').parse.

const { tldExists } = tldjs;

tldExists('google.com');      // returns `true`
tldExists('google.local');    // returns `false` (not an explicit registered TLD)
tldExists('com');             // returns `true`
tldExists('uk');              // returns `true`
tldExists('co.uk');           // returns `true` (because `uk` is a valid TLD)
tldExists('amazon.fancy.uk'); // returns `true` (still because `uk` is a valid TLD)
tldExists('amazon.co.uk');    // returns `true` (still because `uk` is a valid TLD)
tldExists('https://user:[email protected]:8080/some/path?and&query#hash'); // returns `true`

getDomain()

Returns the fully qualified domain from a given string — parseable with require('url').parse.

const { getDomain } = tldjs;

getDomain('google.com');        // returns `google.com`
getDomain('fr.google.com');     // returns `google.com`
getDomain('fr.google.google');  // returns `google.google`
getDomain('foo.google.co.uk');  // returns `google.co.uk`
getDomain('t.co');              // returns `t.co`
getDomain('fr.t.co');           // returns `t.co`
getDomain('https://user:[email protected]:8080/some/path?and&query#hash'); // returns `example.co.uk`

getSubdomain()

Returns the complete subdomain for a given string — parseable with require('url').parse.

const { getSubdomain } = tldjs;

getSubdomain('google.com');             // returns ``
getSubdomain('fr.google.com');          // returns `fr`
getSubdomain('google.co.uk');           // returns ``
getSubdomain('foo.google.co.uk');       // returns `foo`
getSubdomain('moar.foo.google.co.uk');  // returns `moar.foo`
getSubdomain('t.co');                   // returns ``
getSubdomain('fr.t.co');                // returns `fr`
getSubdomain('https://user:[email protected]:443/some/path?and&query#hash'); // returns `secure`

getPublicSuffix()

Returns the public suffix for a given string — parseable with require('url').parse.

const { getPublicSuffix } = tldjs;

getPublicSuffix('google.com');       // returns `com`
getPublicSuffix('fr.google.com');    // returns `com`
getPublicSuffix('google.co.uk');     // returns `co.uk`
getPublicSuffix('s3.amazonaws.com'); // returns `s3.amazonaws.com`
getPublicSuffix('tld.is.unknown');   // returns `unknown`

isValid()

Checks the validity of a given string — parseable with require('url').parse. It does not check if the TLD is well-known.

const { isValid } = tldjs;

isValid('google.com');      // returns `true`
isValid('.google.com');     // returns `false`
isValid('my.fake.domain');  // returns `true`
isValid('localhost');       // returns `false`
isValid('https://user:[email protected]:8080/some/path?and&query#hash'); // returns `true`
isValid('192.168.0.0')      // returns `true`

Troubleshooting

Retrieving subdomain of localhost and custom hostnames

tld.js methods getDomain and getSubdomain are designed to work only with known and valid TLDs. This way, you can trust what a domain is.

localhost is a valid hostname but not a TLD. Although you can instanciate your own flavour of tld.js with additional valid hosts:

const tldjs = require('tldjs');

tldjs.getDomain('localhost');           // returns null
tldjs.getSubdomain('vhost.localhost');  // returns null

const myTldjs = tldjs.fromUserSettings({
  validHosts: ['localhost']
});

myTldjs.getDomain('localhost');           // returns 'localhost'
myTldjs.getSubdomain('vhost.localhost');  // returns 'vhost'

Updating the TLDs List

Many libraries offer a list of TLDs. But, are they up-to-date? And how to update them?

tld.js bundles a list of known TLDs but this list can become outdated. This is especially true if the package have not been updated on npm for a while.

Hopefully for you, even if I'm flying over the world, if I've lost my Internet connection or even if you do manage your own list, you can update it by yourself, painlessly.

How? By passing the --tldjs-update-rules to your npm install command:

# anytime you reinstall your project
npm install --tldjs-update-rules

# or if you add the dependency to your project
npm install --save tldjs --tldjs-update-rules

Open an issue to request an update of the bundled TLDs.

Contributing

Provide a pull request (with tested code) to include your work in this main project. Issues may be awaiting for help so feel free to give a hand, with code or ideas.

Performances

tld.js is fast, but keep in mind that it might vary depending on your own use-case. Because the library tried to be smart, the speed can be drastically different depending on the input (it will be faster if you provide an already cleaned hostname, compared to a random URL).

On an Intel i7-6600U (2,60-3,40 GHz):

For already cleaned hostnames

| Methods | ops/sec | | --- | --- | | isValid | ~8,700,000 | | extractHostname | ~8,100,000 | | tldExists | ~2,000,000 | | getPublicSuffix | ~1,130,000 | | getDomain | ~1,000,000 | | getSubdomain | ~1,000,000 | | parse | ~850,000 |

For random URLs

| Methods | ops/sec | | --- | --- | | isValid | ~25,400,000 | | extractHostname | ~400,000 | | tldExists | ~310,000 | | getPublicSuffix | ~240,000 | | getDomain | ~240,000 | | getSubdomain | ~240,000 | | parse | ~230,000 |

You can measure the performance of tld.js on your hardware by running the following command:

npm run benchmark

Notice: if this is not fast enough for your use-case, keep in mind that you can provide your own extractHostname function (which is the bottleneck in this benchmark) to tld.js.

Contributors

This project exists thanks to all the people who contribute. [Contribute].

Backers

Thank you to all our backers! 🙏 [Become a backer]

Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]

License

MIT License.