npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@wooorm/starry-night

v3.5.0

Published

Syntax highlighting, like GitHub

Downloads

89,711

Vulnerabilities

Powered byPowered by Snyk
  • 0High
  • 0Medium
  • 0Low

Links

Git

Maintainers

wooormwooorm

Keywords

astcodedomgithubgoghhighlighthighlightinglinguistprettylightssyntaxtreelightsvirtual

Readme

starry-night

[![Build][build-badge]][build] [![Coverage][coverage-badge]][coverage] [![Downloads][downloads-badge]][downloads] [![Size][size-badge]][size]

Syntax highlighting, like what GitHub uses to highlight code, but free and open source and JavaScript!

Contents

What is this?

This package is an open source version of GitHub’s closed-source PrettyLights project (more on that later). It supports 600+ grammars and its extremely high quality. It uses TextMate grammars which are also used in popular editors (SublimeText, Atom, VS Code, &c). They’re heavy but high quality.

When should I use this?

starry-night is a high quality highlighter (when your readers or authors are programmers, you want this!) that can support tons of grammars (from new things like MDX to much more!) which approaches how GitHub renders code.

It has a WASM dependency, and rather big grammars, which means that starry-night might be too heavy particularly in browsers, in which case [lowlight][lowlight] or [refractor][refractor] might be more suitable.

This project is similar to the excellent [shiki][shiki], and it uses the same underlying dependencies, but starry-night is meant to match GitHub in that it produces classes and works with the CSS it ships, making it easier to add dark mode and other themes with CSS compared to inline styles.

Finally, this package produces objects (an AST), which makes it useful when you want to perform syntax highlighting in a place where serialized HTML wouldn’t work or wouldn’t work well. For example, when you want to show code in a CLI by rendering to ANSI sequences, when you’re using virtual DOM frameworks (such as React or Preact) so that diffing can be performant, or when you’re working with [hast][hast] or [rehype][rehype].

Bundled, minified, and gzipped, starry-night and the WASM binary are 185 kB. There are two lists of grammars you can use: [common][api-common] (±35 languages, good for your own site) adds 250 kB and [all][api-all] (~600 languages, useful if you are making a site like GitHub) is 1.6 MB. You can also manually choose which grammars to include (or add to common): a language is typically between 3 and 5 kB. To illustrate, Astro costs 2.1 kB and TSX costs 25.4 kB.

What is PrettyLights?

PrettyLights is the syntax highlighter that GitHub uses to turn this:

```markdown
# Hello, world!
```

…into this:

<span class="pl-mh"><span class="pl-mh">#</span><span class="pl-mh"> </span>Hello, world!</span>

…which is what starry-night does too (some small differences in markup, but essentially the same)!

PrettyLights is responsible for taking the flag markdown, looking it up in [languages.yml][languages-yml] from github-linguist to figure out that that means markdown, taking a corresponding grammar (in this case [wooorm/markdown-tm-language][markdown-tm-language]), doing some GPL magic in C, and turning it into spans with classes.

GitHub is using PrettyLights since December 2014, when it [replaced Pygments][ref-1]. They wanted to open source it, but [were unable][ref-2] due to licensing issues. Recently ([Feb 2019][ref-3]?), GitHub has slowly started to move towards TreeLights, which is based on TreeSitter, and also closed source. If TreeLights includes a language (currently: C, C#, CSS, CodeQL, EJS, Elixir, ERB, Gleam, Go, HTML, Java, JS, Nix, PHP, Python, RegEx, Ruby, Rust, TLA, TS), that’ll be used, for everything else PrettyLights is used.

starry-night does what PrettyLights does, not what TreeLights does. I’m hopeful that that will be open sourced in the future and we can mimic both.

[][ref]

Install

This package is [ESM only][esm]. In Node.js (version 16+), install with [npm][]:

npm install @wooorm/starry-night

In Deno with [esm.sh][esmsh]:

import {common, createStarryNight} from 'https://esm.sh/@wooorm/starry-night@3'

In browsers with [esm.sh][esmsh]:

<script type="module">
  import {common, createStarryNight} from 'https://esm.sh/@wooorm/starry-night@3?bundle'
</script>

To get the CSS in browsers, do (see [CSS][] for more info):

<!-- This supports light and dark mode automatically. -->
<link rel="stylesheet" href="https://esm.sh/@wooorm/starry-night@3/style/both">

Use

import {common, createStarryNight} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common)

const scope = starryNight.flagToScope('markdown')
const tree = starryNight.highlight('# hi', scope)

console.log(tree)

Yields:

{
  type: 'root',
  children: [
    {
      type: 'element',
      tagName: 'span',
      properties: {className: ['pl-mh']},
      children: [
        {type: 'text', value: '# '},
        {
          type: 'element',
          tagName: 'span',
          properties: {className: ['pl-en']},
          children: [{type: 'text', value: 'hi'}]
        }
      ]
    }
  ]
}

API

This package exports the identifiers [all][api-all], [common][api-common], and [createStarryNight][api-create-starry-night] from the main module. There is no default export.

It also includes grammars directly in its export map. Do not use the lang/ folder or the .js extension. For CSS files, do use style/ but don’t use .css:

import mdx from '@wooorm/starry-night/source.mdx' // Grammar.
import tritanopiaDark from '@wooorm/starry-night/style/tritanopia-dark' // CSS.

all

List of all grammars ([Array<Grammar>][api-grammar])

common

List of ±35 common grammars ([Array<Grammar>][api-grammar])

createStarryNight(grammars[, options])

Create a StarryNight that can highlight things with the given grammars. This is async to allow async loading and registering, which is currently only used for WASM.

Parameters
  • grammars ([Array<Grammar>][api-grammar]) — grammars to support
  • options ([Options][api-options], optional) — configuration
Returns

Promise that resolves to an instance which highlights with the bound grammars (Promise<StarryNight>).

starryNight.flagToScope(flag)

Get the grammar scope (such as text.md) associated with a grammar name (such as markdown) or grammar extension (such as .mdwn).

This function uses the first word (when splitting on spaces and tabs) that is used after the opening of a fenced code block:

```js
console.log(1)
```

To match GitHub, this also accepts entire paths:

```path/to/example.js
console.log(1)
```

👉 Note: languages can use the same extensions. For example, .h is reused by many languages. In those cases, you will get one scope back, but it might not be the most popular language associated with an extension.

Parameters
  • flag (string) — grammar name (such as 'markdown'), grammar extension (such as '.mdwn'), or entire file path ending in extension
Returns

Grammar scope, such as 'text.md' (string or undefined).

Example
import {common, createStarryNight} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common)

console.log(starryNight.flagToScope('pandoc')) // `'text.md'`
console.log(starryNight.flagToScope('workbook')) // `'text.md'`
console.log(starryNight.flagToScope('.workbook')) // `'text.md'`
console.log(starryNight.flagToScope('path/to/example.js')) // `'source.js'`
console.log(starryNight.flagToScope('whatever')) // `undefined`

starryNight.highlight(value, scope)

Highlight programming code.

Parameters
  • value (string) — code to highlight
  • scope (string) — registered grammar scope to highlight as (such as 'text.md')
Returns

Node representing highlighted code ([Root][hast-root]).

Example
import {createStarryNight} from '@wooorm/starry-night'
import sourceCss from '@wooorm/starry-night/source.css'

const starryNight = await createStarryNight([sourceCss])

console.log(starryNight.highlight('em { color: red }', 'source.css'))

Yields:

{
  type: 'root',
  children: [
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]},
    {type: 'text', value: ' { '},
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]},
    {type: 'text', value: ': '},
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]},
    {type: 'text', value: ' }'}
  ]
}

starryNight.missingScopes()

List scopes that are needed by the registered grammars but that are missing.

To illustrate, the text.xml.svg grammar needs the text.xml grammar. When you register text.xml.svg without text.xml, it will be listed here.

Returns

List of grammar scopes, such as 'text.md' (Array<string>).

Example
import {createStarryNight} from '@wooorm/starry-night'
import textXml from '@wooorm/starry-night/text.xml'
import textXmlSvg from '@wooorm/starry-night/text.xml.svg'

const svg = await createStarryNight([textXmlSvg])
console.log(svg.missingScopes()) //=> ['text.xml']

const svgAndXml = await createStarryNight([textXmlSvg, textXml])
console.log(svgAndXml.missingScopes()) //=> []

starryNight.register(grammars)

Add more grammars.

Parameters
  • grammars ([Array<Grammar>][api-grammar]) — grammars to support
Returns

Promise resolving to nothing (Promise<undefined>).

Example
import {createStarryNight} from '@wooorm/starry-night'
import sourceCss from '@wooorm/starry-night/source.css'
import textMd from '@wooorm/starry-night/text.md'
import {toHtml} from 'hast-util-to-html'

const markdown = '```css\nem { color: red }\n```'

const starryNight = await createStarryNight([textMd])

console.log(toHtml(starryNight.highlight(markdown, 'text.md')))

await starryNight.register([sourceCss])

console.log(toHtml(starryNight.highlight(markdown, 'text.md')))

Yields:

<span class="pl-s">```</span><span class="pl-en">css</span>
<span class="pl-c1">em { color: red }</span>
<span class="pl-s">```</span>
<span class="pl-s">```</span><span class="pl-en">css</span>
<span class="pl-ent">em</span> { <span class="pl-c1">color</span>: <span class="pl-c1">red</span> }
<span class="pl-s">```</span>

starryNight.scopes()

List all registered scopes.

Returns

List of grammar scopes, such as 'text.md' (Array<string>).

Example
import {common, createStarryNight} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common)

console.log(starryNight.scopes())

Yields:

[
  'source.c',
  'source.c++',
  // …
  'text.xml',
  'text.xml.svg'
]

GetOnigurumaUrl

Function to get a URL to the oniguruma WASM (TypeScript type).

👉 Note: this must currently result in a version 2 URL of onig.wasm from [vscode-oniguruma][vscode-oniguruma].

⚠️ Danger: when you use this functionality, your project might break at any time (when reinstalling dependencies), except when you make sure that the WASM binary you load manually is what our internally used vscode-oniguruma dependency expects. To solve this, you could for example use an npm script called [dependencies][npm-script-dependencies] (which runs everytime node_modules is changed) which copies vscode-oniguruma/release/onig.wasm to the place you want to host it.

Returns

URL object to a WASM binary (Promise<URL> or URL).

Example
import {common, createStarryNight} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common, {
  getOnigurumaUrlFetch() {
    return new URL('/onig.wasm', window.location.href);
  }
})

Grammar

TextMate grammar with some extra info (TypeScript type).

Fields
  • dependencies (Array<string>, optional, example: ['source.tsx']) — list of scopes that are needed for this grammar to work
  • extensions (Array<string>, example: ['.mdx']) — list of extensions
  • extensionsWithDot (Array<string>, optional, example: ['.php']) — list of extensions that only match if used w/ a dot
  • injections (Record<string, Rule>, optional) — TextMate injections
  • names (Array<string>, example: ['mdx']) — list of names
  • patterns (Array<Rule>) — TextMate patterns
  • repository (Record<string, Rule>, optional) — TextMate repository
  • scopeName (string, example: 'source.mdx') — scope

Options

Configuration (TypeScript type).

Fields
  • getOnigurumaUrlFetch ([GetOnigurumaUrl][api-get-oniguruma-url], optional) — get a URL to the oniguruma WASM, typically used in browsers
  • getOnigurumaUrlFs ([GetOnigurumaUrl][api-get-oniguruma-url], optional) — get a URL to the oniguruma WASM, typically used in Node.js

Examples

Example: serializing hast as html

[hast][hast] trees as returned by starry-night can be serialized with [hast-util-to-html][hast-util-to-html]:

import {common, createStarryNight} from '@wooorm/starry-night'
import {toHtml} from 'hast-util-to-html'

const starryNight = await createStarryNight(common)

const tree = starryNight.highlight('"use strict";', 'source.js')

console.log(toHtml(tree))

Yields:

<span class="pl-s"><span class="pl-pds">"</span>use strict<span class="pl-pds">"</span></span>;

Example: using starry-night on the client

You don’t have to do preprocess things on a server. Particularly, when you are not using Node.js or so. Or, when you have a lot of often changing content (likely markdown), such as on a page of comments.

In those cases, you can run starry-night in the browser. Here is an example. It also uses [hast-util-to-dom][hast-util-to-dom], which is a light way to turn the AST into DOM nodes.

Say we have this example.js on our browser (no bundling needed!):

import {
  common,
  createStarryNight
} from 'https://esm.sh/@wooorm/starry-night@3?bundle'
import {toDom} from 'https://esm.sh/hast-util-to-dom@4?bundle'

const starryNight = await createStarryNight(common)
const prefix = 'language-'

const nodes = Array.from(document.body.querySelectorAll('code'))

for (const node of nodes) {
  const className = Array.from(node.classList).find(function (d) {
    return d.startsWith(prefix)
  })
  if (!className) continue
  const scope = starryNight.flagToScope(className.slice(prefix.length))
  if (!scope) continue
  const tree = starryNight.highlight(node.textContent, scope)
  node.replaceChildren(toDom(tree, {fragment: true}))
}

…and then, if we would have an index.html for our document:

<!doctype html>
<meta charset=utf8>
<title>Hello</title>
<link rel=stylesheet href=https://esm.sh/@wooorm/starry-night@3/style/both>
<body>
<h1>Hello</h1>
<p>…world!</p>
<pre><code class=language-js>console.log('it works!')
</code></pre>
<script type=module src=./example.js></script>
</body>

Opening that page in a browser, we’d see the <code> being swapped with:

<code class="language-js"><span class="pl-en">console</span>.<span class="pl-c1">log</span>(<span class="pl-s"><span class="pl-pds">'</span>it works!<span class="pl-pds">'</span></span>)
</code>

Example: turning hast into react nodes

hast trees as returned by starry-night can be turned into preact, react, solid, svelte, vue, etc., with [hast-util-to-jsx-runtime][hast-util-to-jsx-runtime]:

import {common, createStarryNight} from '@wooorm/starry-night'
import {toJsxRuntime} from 'hast-util-to-jsx-runtime'
import {Fragment, jsx, jsxs} from 'react/jsx-runtime'

const starryNight = await createStarryNight(common)

const tree = starryNight.highlight('"use strict";', 'source.js')
const reactNode = toJsxRuntime(tree, {Fragment, jsx, jsxs})

console.log(reactNode)

Yields:

{
  '$$typeof': Symbol(react.element),
  type: Symbol(react.fragment),
  key: null,
  ref: null,
  props: { children: [ [Object], ';' ] },
  _owner: null,
  _store: {}
}

Example: adding line numbers

GitHub itself does not add line numbers to the code they highlight. You can do that, by transforming the AST. Here’s an example of a utility that wraps each line into a span with a class and a data attribute with its line number. That way, you can style the lines as you please. Or you can generate different elements for each line, of course.

Say we have our utility as hast-util-starry-night-gutter.js:

/**
 * @import {ElementContent, Element, RootContent, Root} from 'hast'
 */

/**
 * @param {Root} tree
 *   Tree.
 * @returns {undefined}
 *   Nothing.
 */
export function starryNightGutter(tree) {
  /** @type {Array<RootContent>} */
  const replacement = []
  const search = /\r?\n|\r/g
  let index = -1
  let start = 0
  let startTextRemainder = ''
  let lineNumber = 0

  while (++index < tree.children.length) {
    const child = tree.children[index]

    if (child.type === 'text') {
      let textStart = 0
      let match = search.exec(child.value)

      while (match) {
        // Nodes in this line.
        const line = /** @type {Array<ElementContent>} */ (
          tree.children.slice(start, index)
        )

        // Prepend text from a partial matched earlier text.
        if (startTextRemainder) {
          line.unshift({type: 'text', value: startTextRemainder})
          startTextRemainder = ''
        }

        // Append text from this text.
        if (match.index > textStart) {
          line.push({
            type: 'text',
            value: child.value.slice(textStart, match.index)
          })
        }

        // Add a line, and the eol.
        lineNumber += 1
        replacement.push(createLine(line, lineNumber), {
          type: 'text',
          value: match[0]
        })

        start = index + 1
        textStart = match.index + match[0].length
        match = search.exec(child.value)
      }

      // If we matched, make sure to not drop the text after the last line ending.
      if (start === index + 1) {
        startTextRemainder = child.value.slice(textStart)
      }
    }
  }

  const line = /** @type {Array<ElementContent>} */ (tree.children.slice(start))
  // Prepend text from a partial matched earlier text.
  if (startTextRemainder) {
    line.unshift({type: 'text', value: startTextRemainder})
    startTextRemainder = ''
  }

  if (line.length > 0) {
    lineNumber += 1
    replacement.push(createLine(line, lineNumber))
  }

  // Replace children with new array.
  tree.children = replacement
}

/**
 * @param {Array<ElementContent>} children
 * @param {number} line
 * @returns {Element}
 */
function createLine(children, line) {
  return {
    type: 'element',
    tagName: 'span',
    properties: {className: 'line', dataLineNumber: line},
    children
  }
}

…and a module example.js:

import {common, createStarryNight} from '@wooorm/starry-night'
import {toHtml} from 'hast-util-to-html'
import {starryNightGutter} from './hast-util-starry-night-gutter.js'

const starryNight = await createStarryNight(common)

const tree = starryNight.highlight(
  '# Some heading\n\n```js\nalert(1)\n```\n***',
  'text.md'
)

starryNightGutter(tree)

console.log(toHtml(tree))

Now running node example.js yields:

<span class="line" data-line-number="1"><span class="pl-mh"># <span class="pl-en">Some heading</span></span></span>
<span class="line" data-line-number="2"></span>
<span class="line" data-line-number="3"><span class="pl-s">```</span><span class="pl-en">js</span></span>
<span class="line" data-line-number="4"><span class="pl-en">alert</span>(<span class="pl-c1">1</span>)</span>
<span class="line" data-line-number="5"><span class="pl-s">```</span></span>
<span class="line" data-line-number="6"><span class="pl-ms">***</span></span>

Example: integrate with unified, remark, and rehype

This example shows how to use [rehype-starry-night][rehype-starry-night] with [unified][unified]. If we have a markdown file example.md:

# Hello

…world!

```js
console.log('it works!')
```

…and a module example.js:

import fs from 'node:fs/promises'
import rehypeStarryNight from 'rehype-starry-night'
import rehypeStringify from 'rehype-stringify'
import remarkParse from 'remark-parse'
import remarkRehype from 'remark-rehype'
import {unified} from 'unified'

const file = await unified()
  .use(remarkParse)
  .use(remarkRehype)
  .use(rehypeStarryNight)
  .use(rehypeStringify)
  .process(await fs.readFile('example.md'))

console.log(String(file))

…then running node example.js yields:

<h1>Hello</h1>
<p>…world!</p>
<pre><code class="language-js"><span class="pl-en">console</span>.<span class="pl-c1">log</span>(<span class="pl-s"><span class="pl-pds">'</span>it works!<span class="pl-pds">'</span></span>)
</code></pre>

Example: integrating with markdown-it

This example shows how to combine starry-night with markdown-it. If we have a markdown file example.md:

# Hello

…world!

```js
console.log('it works!')
```

…and a module example.js:

/**
 * @import {ElementContent} from 'hast'
 */

import fs from 'node:fs/promises'
import {common, createStarryNight} from '@wooorm/starry-night'
import {toHtml} from 'hast-util-to-html'
import markdownIt from 'markdown-it'

const file = await fs.readFile('example.md')
const starryNight = await createStarryNight(common)

const markdownItInstance = markdownIt({
  highlight(value, lang) {
    const scope = starryNight.flagToScope(lang)

    return toHtml({
      type: 'element',
      tagName: 'pre',
      properties: {
        className: scope
          ? [
              'highlight',
              'highlight-' + scope.replace(/^source\./, '').replace(/\./g, '-')
            ]
          : undefined
      },
      children: scope
        ? /** @type {Array<ElementContent>} */ (
            starryNight.highlight(value, scope).children
          )
        : [{type: 'text', value}]
    })
  }
})

const html = markdownItInstance.render(String(file))

console.log(html)

Now running node example.js yields:

<h1>Hello</h1>
<p>…world!</p>
<pre class="highlight highlight-js"><span class="pl-en">console</span>.<span class="pl-c1">log</span>(<span class="pl-s"><span class="pl-pds">'</span>it works!<span class="pl-pds">'</span></span>)
</pre>

[][ref]

Syntax tree

The generated [hast][hast] starts with a root node, that represents the fragment. It contains up to three levels of <span> elements, each with a single class. All these levels can contain text nodes with the actual code. Interestingly, TextMate grammars work per line, so all line endings are in the root directly, meaning that creating a gutter to display line numbers can be generated rather naïvely by only looking through the root node.

CSS

starry-night does not inject CSS for the syntax highlighted code (because well, starry-night doesn’t have to be turned into HTML and might not run in a browser!). If you are in a browser, you can use the packaged themes, or get creative with CSS! 💅

All themes accept CSS variables (custom properties). With the theme core.css, you have to define your own properties. All other themes define the colors on :root. Themes either have a dark or light suffix, or none, in which case they automatically switch colors based on a @media (prefers-color-scheme: dark). All themes are tiny (under 1 kB). The shipped themes are as follows:

| name | Includes light scheme | Includes dark scheme | | - | - | - | | @wooorm/starry-night/style/core | | | | @wooorm/starry-night/style/light | ✅ | | | @wooorm/starry-night/style/dark | | ✅ | | @wooorm/starry-night/style/both | ✅ | ✅ | | @wooorm/starry-night/style/colorblind-light | ✅ | | | @wooorm/starry-night/style/colorblind-dark | | ✅ | | @wooorm/starry-night/style/colorblind | ✅ | ✅ | | @wooorm/starry-night/style/dimmed-dark | | ✅ | | @wooorm/starry-night/style/dimmed | ✅ | ✅ | | @wooorm/starry-night/style/high-contrast-light | ✅ | | | @wooorm/starry-night/style/high-contrast-dark | | ✅ | | @wooorm/starry-night/style/high-contrast | ✅ | ✅ | | @wooorm/starry-night/style/tritanopia-light | ✅ | | | @wooorm/starry-night/style/tritanopia-dark | | ✅ | | @wooorm/starry-night/style/tritanopia | ✅ | ✅ |

Languages

Checked grammars are included in [common][api-common]. Everything (that’s needed) is available through [all][api-all]. You can add more grammars as you please.

Each grammar has several associated names and extensions. See source files for which are known and use flagToScope to turn them into scopes.

Some grammars need other grammars to work. You are responsible for loading those, use missingScopes to find which dependencies are needed.

All licenses are permissive and made available in [notice][notice]. Changes should go to upstream repos and [languages.yml][languages-yml] in github-linguist.