@reuben-chamsys5/lunr-extension
v1.0.0-alpha.49
Published
An Antora extension that adds offline, full-text search powered by Lunr to your Antora documentation site.
Downloads
13
Maintainers
Readme
Antora Lunr Extension
Lunr provides a great search experience for readers without having to rely on external search services. With this extension, you can add offline, full-text search powered by Lunr to your Antora documentation site.
📌 NOTE: The Antora Lunr Extension is designed for and compatible with Antora 3.0 and newer. If you're using an earlier version of Antora, you must use antora-lunr instead.
To add search functionality powered by Lunr to your Antora documentation site, you need to install this extension package, register the extension in your Antora playbook file, and add the search interface to the pages in your site. Let's get started.
Prerequisites
In order to use this extension, you must be using at least Node.js 16 and Antora 3. We assume you've already set up an Antora playbook file (i.e., antora-playbook.yml) to build your site.
Installation
Begin by installing the extension package into your playbook project:
$ npm i @antora/lunr-extension
You also have the option of installing the extension globally by adding the -g
flag to the npm i
command:
$ npm i -g @antora/lunr-extension
We strongly recommend installing dependencies into your playbook project. This strategy makes it easier to manage the dependencies, clearly documents those dependencies, and ensures the build is portable and isolated from other sites.
Both of the previous two commands download and install the latest release of this software from the npm registry. If you want to use the development version, please refer to the <<contributing, contributing section>>.
Although the software in the git repository is regularly and rigorously tested, the behavior of the development version may not always match the documentation.
Usage
This section explains how to activate the Lunr integration and how to use the search interface it provides.
Register the extension
Now that you have the Lunr extension installed, you need to register the extension with Antora.
To register the extension, you'll add an entry that cites the name of the package to the antora.extensions
key in your Antora playbook file.
Open the Antora playbook file and add the extension as follows:
antora-playbook.yml
antora:
extensions:
- '@antora/lunr-extension'
If you don't need to specify any configuration keys, the entry in antora.extensions
can be the name of the package.
In this case, quotes are required around the package name since @
is a special character in YAML.
💡 TIP: Alternately, you can register the extension when you run the antora
command using the --extension
option.
In order to specify configuration keys for the extension, you must change the entry to a map syntax.
When using the map syntax, the package name must be preceded by the require
key, as shown here:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
You may want to start with this syntax so you don't have to remember to switch to it later when you want to specify configuration.
Generate an index file
The next time you build your documentation site, the extension will automatically generate a search index and write it to a file named search-index.js at the root of the site.
The location of this file depends on the value of output.dir
key in your playbook.
When using the default output dir, that location is build/site/search-index.js.
Set (or don't set) the site URL
The search is not coupled to the value of the site URL. That means you can set the site URL in the playbook to any allowable value and the search will work regardless. (See the docs for the site.url key for a list of allowable values). In fact, you don't have to set the site URL at all. This works because the URLs in the search results are always computed relative to the current page. They are not absolute URLs.
💡 TIP: If you're using the http-server module to provide an HTTP server to view your site locally, you can set the site.url
key to http://localhost:8080
to emulate the conditions of a production environment.
Thanks to the use of relative URLs in the search results, the browser is able to resolve URLs in the results regardless of where the site is deployed or on what page the search is used. You can even use the search when viewing the site offline through a file URI.
Generate the site
If you registered the extension in your playbook file, you can generate your site using the antora
command without having to pass any additional options or environment variables.
$ antora antora-playbook.yml
If you have not registered the extension in your playbook file, you can register it using the --extension
CLI option of the antora
command:
$ antora --extension @antora/lunr-extension antora-playbook.yml
Using the --extension
option also allows you to enable the extension that's registered in the playbook file, but marked as not enabled using the enabled
key.
See Enable an Extension for details about how that works.
Searching
Once you have incorporated the Lunr-based search into your site, the next step is to perform a search.
To search, start typing a query (i.e., a word or phrase) into the search box at the top of the page. For example:
install
The search results will be shown under the search box in real time as you type.
The search client will first attempt to find an exact match.
If that query doesn't return any results, the search client will try a begins with search (e.g., install*
).
If that query doesn't return any results, the search client will try a contains search (e.g., *install*
).
If no results are found, the search results will report “No results found for query”.
📌 NOTE: The search engine applies stemming to terms, so it may also find matches for different forms of the word that have the same meaning (i.e., root words).
Lunr supports searches for multiple terms. However, the default behavior may not match your experience using other search engines. By default, multiple search terms are combined with an OR operator (i.e., either term). Consider the following query:
install antora
If a document matches at least one of the search terms, the document will show up in the results. Documents that contain both terms will score higher and thus show up first.
If you want to search for documents that have all of the terms entered (i.e., required term presence), you must add a modifier to the terms.
To indicate that a term must be present in a matching documents, prefix the term with +
(e.g., +install
).
For example, to find a document that contains both “install” and “antora”, use the following query:
+install +antora
🔥 CAUTION: Unlike other search engines, Lunr does not support queries that are enclosed in quotes. In the future, this extension may automatically translate a query with that syntax into a query that uses required term presence.
If you only want to search for a term in page titles and section titles, prefix the term with title:
.
For example:
title:install
When searching for titles, only titles will show in the results. You will not see text snippets from other areas of the page.
You can search for titles that have all of the specified terms by combining the title:
prefix with the +
modifier.
For example:
+title:install +title:antora
The results can be further filtered. Next to the results you can find a filter panel. In this panel, you can select all components to which the search should be limited. For each selected component you also can select a specific version to which the search should be limited. If you don't need these filters you can <<additional-filters-configuration,disable them by configuration>>.
This extension will likely support more advanced searches in the future, so watch this space to learn about new searching capabilities.
To learn more about how Lunr searches work, see Searching with Lunr.
Loading the search index
Depending on the index size, it can take a few seconds to download and load a pre-built index. While loading the pre-built index, the search input will be disabled. Once the index is loaded, the search input will become active and the user will be able to perform a search.
In addition, an event will be published on the search input element. You can add an event listener if you want to be notified when the index is loaded:
const searchInput = document.getElementById('search-input')
searchInput.addEventListener('loadedindex', (event) => {
console.log(`Index loaded in: ${event.detail.took}ms`) // <1>
})
<1> The event contains the time spent loading the index in milliseconds
Configuration
This section explains how to control the behavior of this integration using configuration keys and other settings.
In order to specify configuration keys, the entry in the antora.extensions
key must use the map syntax.
When using this form, you specify the package name using the require
key.
The configuration keys should be siblings of the require
key.
Only the require
key should be prefixed with the entry marker, -
.
📌 NOTE: In antora-lunr (the predecessor of this extension), configuration was performed using environment variables. In this extension, configuration is now done using configuration keys in the playbook.
Index only the latest version
To index only the latest (i.e., released) version, set the index_latest_only
configuration key:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
index_latest_only: true
By default the extension indexes all the versions of your documentation components.
Additional filters
You can disable the <<additional-filters-description, additional filters>>.
To disable the additional filters, set the additional_filters
configuration key:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
additional_filters: false
Exclude pages
You can instruct the indexer to exclude certain pages by defining the noindex
document attribute in the AsciiDoc header:
# Do Not Index Me
This content will not show up in the search results.
You can exclude the pages for an entire component version by defining the noindex
AsciiDoc attribute in the component descriptor for that version:
antora.yml
asciidoc:
attributes:
noindex: '@'
The indexer will also look for the robots
meta tag in HTML document and exclude the page if the value of the content attribute is noindex
.
That allows you to exclude pages that were either not created from AsciiDoc or that contain a meta robots tag that was added by the UI template based on another condition.
Snippet length
By default, the maximum length of a snippet in a search result is 100 characters.
To change this length, set the snippet_length
configuration key:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
snippet_length: 250
The value of this option is an upper limit. The length of the snippet will not exceed the length of the matched text.
Support for other languages
By default, Lunr only supports English as an indexing language. You can add support for the following other languages:
- Arabic (ar)
- Chinese (zh) (see note below)
- Danish (da)
- Dutch (nl)
- Finnish (fi)
- French (fr)
- German (de)
- Hindi (hi)
- Hungarian (hu)
- Italian (it)
- Japanese (ja)
- Norwegian (no)
- Portuguese (pt)
- Romanian (ro)
- Russian (ru)
- Spanish (es)
- Swedish (sv)
- Thai (th)
- Turkish (tr)
- Vietnamese (vi)
📌 NOTE: To use Chinese as your language, you must install the @node-rs/jieba
dependency (i.e., npm i @node-rs/jieba
).
To use one or more languages, set the languages
configuration key with all the desired language codes as a list:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
languages: [en, fr]
Configuring additional stop words
By default Lunr provides a list of stop words in English which are common words that are excluded from the search index. In addition, Lunr Languages provides a similar stop word list filter for other supported languages.
It is possible to specify additional stop words using the extra_stop_words
key in a playbook.
You can define this key globally so it applies to all languages:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
extra_stop_words: ['basically', 'actually']
You can also define extra_stop_words
on language by language basis:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
langauges:
- en:
extra_stop_words: [''basically', 'actually']
- fr:
extra_stop_words: ['empêche', 'fonctionnalité']
You do not have to define the extra_stop_words
key for a language if you do not wish to specify stop words for it.
For example, to add Spanish as an additional indexing language with no extra stop words in the example above you would use:
antora-playbook.yml
antora:
extensions:
- require: '@antora/lunr-extension'
langauges:
- en:
extra_stop_words: [''basically', 'actually']
- fr:
extra_stop_words: ['empêche', 'fonctionnalité']
- es
Debug invalid queries
By default, the search client will silence the error thrown by Lunr if a query is invalid. The client assumes that if the query is invalid, there are no results to present.
If you're trying to debug a query, and you want to see the message of the error Lunr throws, you can put the search client into debug mode.
To do so, add ?lunr-debug=1
to the end of the URL of the current page and reload the page.
💡 TIP: If the page URL already has a query string, then append &lunr-debug=1
instead.
When debug mode is enabled, messages about invalid queries will show up in the browser console at the Debug level. For example:
Invalid search query: invalid:term (unrecognised field 'invalid', possible fields: 'title', 'name', 'text', 'component')
The user is not currently notified about an invalid query, though that could change in the future.
UI assumptions
This section is intended for anyone designing and creating a custom UI. This extension relies on a contract with the UI in order to minimize the configuration the user must perform to get the extension working. Antora's default UI fulfills this contract. For custom UIs, the assumptions of this contract are documented here.
lunr.js script
The extension provides the lunr.js script. Your UI should not include it. If it does, you will get a duplicate file error.
Environment variable
When this extension is enabled, it sets the SITE_SEARCH_PROVIDER
environment variable to the value lunr
.
This variable is available to the UI templates as env.SITE_SEARCH_PROVIDER
.
The existence of this variable informs the UI template which search integration is active (in this case, Lunr).
When this variable is set, the UI is expected to add certain elements to support the extension.
📌 NOTE: If the UI you're using does not fulfill this contract, you'll need to use the supplemental UI to complete the contract.
Search input
This extension assumes that the UI will add an input field for search somewhere in the page. Currently, the provided styles assume it's located in the navbar. The template snippet should look something like this:
{{#if env.SITE_SEARCH_PROVIDER}}
<input id="search-input" type="text" placeholder="Search the docs">
{{/if}}
The UI may enclose the input in other elements in order to position it properly. In Antora's default UI, it looks like this:
{{#if env.SITE_SEARCH_PROVIDER}}
<div class="navbar-item search hide-for-print">
<div id="search-field" class="field">
<input id="search-input" type="text" placeholder="Search the docs"{{#if page.home}} autofocus{{/if}}>
</div>
</div>
{{/if}}
The only requirement is that the input be of type text
and have the ID search-input
.
Restrict search to current component
You can use the following snippet if you want to add a checkbox in order to restrict searches to the current component:
{{#if env.SITE_SEARCH_PROVIDER}}
<div class="navbar-item search hide-for-print">
<div id="search-field" class="field has-filter">
<input id="search-input" type="text" placeholder="Search the docs"{{#if page.home}} autofocus{{/if}}>
<label class="filter checkbox">
<input type="checkbox" data-facet-filter="component:{{page.component.name}}"> In this project
</label>
</div>
</div>
{{/if}}
If you want to filter on the current component by default, you can add the checked
attribute on the checkbox:
<input type="checkbox" data-facet-filter="component:{{page.component.name}}" checked> In this project
If you always want to filter on the current component (without giving an option), you can hide the checkbox using an inline style:
<label class="filter checkbox" style="display: none">
<input type="checkbox" data-facet-filter="component:{{page.component.name}}" checked> In this project
</label>
Search scripts
This extension assumes that the UI will include the search-scripts partial somewhere in the footer. This partial loads the lunr script, search UI script, and search index into the page. The template snippet that includes this partial should look something like this:
{{#if env.SITE_SEARCH_PROVIDER}}
{{> search-scripts}}
{{/if}}
This extension will automatically add the partial search-scripts to the UI catalog if it does not already exist.
Search styles
This package provides additional CSS to style the search results (data/css/search.css). This stylesheet is loaded by the search UI script also provided from this package. This extension will automatically add the stylesheet search.css to the UI catalog if it does not already exist.
If a custom UI depends on additional styles, the creator of the UI can either bundle those styles or ask the user to place an alternate stylesheet in the supplemental UI.
Contributing
If you are interested in contributing to this project, please refer to the <<contributing.adoc#,contributing guide>>. In this guide, you'll learn how to:
- <<contributing.adoc#set-up-workspace,set up your development workspace>>
- <<contributing.adoc#build-project,build the project>>
- <<contributing.adoc#run-dev-version,run the development version>>
- <<contributing.adoc#project-rq,submit a merge request>>
Thanks in advance for helping to make this project a success!
Who's using it?
Here's a list of projects using the Antora Lunr extension.
- SUSE Manager Documentation
- Uyuni Documentation
- Beat Link Trigger
- NTT Application Security
- Commodore Components Hub (VSHN)
- DHW - DataHandwerk toolkit mssql
- Apache Solr Reference Guide
To add your project to this list, please edit this file!
Trademarks
AsciiDoc(R) is a trademark of the Eclipse Foundation, Inc.