npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

retext-keywords

v8.0.2

Published

retext plugin to extract keywords

Downloads

5,132

Readme

retext-keywords

Build Coverage Downloads Size Sponsors Backers Chat

retext plugin to extract keywords and key phrases.

Contents

What is this?

This package is a unified (retext) plugin to extract keywords and key phrases from a document, and exposes that metadata on the file.

When should I use this?

You can use this plugin any time you’re dealing with unified or retext already, and are interested in keywords and key phrases. Importantly, keywords extraction in NLP is a rather heavy and sometimes fragile process, so you might be better off manually providing a list of keywords.

Install

This package is ESM only. In Node.js (version 16+), install with npm:

npm install retext-keywords

In Deno with esm.sh:

import retextKeywords from 'https://esm.sh/retext-keywords@8'

In browsers with esm.sh:

<script type="module">
  import retextKeywords from 'https://esm.sh/retext-keywords@8?bundle'
</script>

Use

Say our document example.txt contains (from Wikipedia):

Terminology mining, term extraction, term recognition, or glossary extraction, is a subtask of information extraction. The goal of terminology extraction is to automatically extract relevant terms from a given corpus.

In the semantic web era, a growing number of communities and networked enterprises started to access and interoperate through the internet. Modeling these communities and their information needs is important for several web applications, like topic-driven web crawlers, web services, recommender systems, etc. The development of terminology extraction is essential to the language industry.

One of the first steps to model the knowledge domain of a virtual community is to collect a vocabulary of domain-relevant terms, constituting the linguistic surface manifestation of domain concepts. Several methods to automatically extract technical terms from domain-specific document warehouses have been described in the literature.

Typically, approaches to automatic term extraction make use of linguistic processors (part of speech tagging, phrase chunking) to extract terminological candidates, i.e. syntactically plausible terminological noun phrases, NPs (e.g. compounds "credit card", adjective-NPs "local tourist information office", and prepositional-NPs "board of directors" - in English, the first two constructs are the most frequent). Terminological entries are then filtered from the candidate list using statistical and machine learning methods. Once filtered, because of their low ambiguity and high specificity, these terms are particularly useful for conceptualizing a knowledge domain or for supporting the creation of a domain ontology. Furthermore, terminology extraction is a very useful starting point for semantic similarity, knowledge management, human translation and machine translation, etc.

…and our module example.js contains:

import {toString} from 'nlcst-to-string'
import {retext} from 'retext'
import retextKeywords from 'retext-keywords'
import retextPos from 'retext-pos'
import {read} from 'to-vfile'

const file = await retext()
  .use(retextPos) // Make sure to use `retext-pos` before `retext-keywords`.
  .use(retextKeywords)
  .process(await read('example.txt'))

console.log('Keywords:')

if (file.data.keywords) {
  for (const keyword of file.data.keywords) {
    console.log(toString(keyword.matches[0].node))
  }
}

console.log()
console.log('Key-phrases:')

if (file.data.keyphrases) {
  for (const phrase of file.data.keyphrases) {
    console.log(toString(phrase.matches[0].nodes))
  }
}

…then running node example.js yields:

Keywords:
term
extraction
Terminology
web
domain

Key-phrases:
terminology extraction
terms
term extraction
knowledge domain
communities

API

This package exports no identifiers. The default export is retextKeywords.

unified().use(retextKeywords[, options])

Extract keywords and key phrases.

The results are stored on file.data.keyphrases (Array<Keyphrase>) and file.data.keywords (Array<Keyword>).

Parameters
  • options (Options, optional) — configuration
Returns

Transform (Transformer).

Keyphrase

Info on a key phrase (TypeScript type).

Fields
  • matches (Array<PhraseMatch>) — all matches
  • score (number) — score of phrase, for one match
  • stems (Array<string>) — stems of phrase
  • weight (number) — score of phrase, for all matches

Keyword

Info on a keyword (TypeScript type).

Fields
  • matches (Array<WordMatch>) — all matches
  • score (number) — score of word, for all matches
  • stem (string) — stems of word

Options

Configuration (TypeScript type).

Fields
  • maximum (number, default: 5) — try to detect at most maximum words and phrases; actual counts may differ, for example, when two words have the same score, both will be returned; when too few words exist, less will be returned

PhraseMatch

Match (TypeScript type).

Fields

WordMatch

Match (TypeScript type).

Fields
  • node (Node) — matched node
  • index (number) — index of node in parent
  • parent (Node) — parent

Types

This package is fully typed with TypeScript. It exports the additional types Keyphrase, Keyword, Options, PhraseMatch, and WordMatch.

It also registers the file.data fields with vfile. If you’re working with the file, make sure to import this plugin somewhere in your types, as that registers the new fields on the file.

/**
 * @import {} from 'retext-keywords'
 */

import {VFile} from 'vfile'

const file = new VFile()

console.log(file.data.keywords) //=> TS now knows this is `Array<Keyword> | undefined`.

Compatibility

Projects maintained by the unified collective are compatible with maintained versions of Node.js.

When we cut a new major release, we drop support for unmaintained versions of Node. This means we try to keep the current release line, retext-keywords@^8, compatible with Node.js 16.

Contribute

See contributing.md in retextjs/.github for ways to get started. See support.md for ways to get help.

This project has a code of conduct. By interacting with this repository, organization, or community you agree to abide by its terms.

License

MIT © Titus Wormer