MathJax Renderer

This is the math renderer used in webapp as of November 2023. It relies on MathJax 3.2.2 to do the heavy lifting of converting TeX to HTML, accessible MathML, and CSS. Most of the code in this repo is "adapter" code that fixes up TeX in our content (originally written for KaTeX) so it works with MathJax. For additional documentation, please check out the Confluence page MathJax at Khan Academy.

Installation

Find the current version number in package.json.

Then, add the package as a dependency of your project, specifying the version in the URL hash:

# Assuming you want version v1.0.3:
yarn add 'git+ssh://[email protected]:Khan/mathjax-renderer.git#v1.0.3'

Usage

import {MathJaxRenderer} from "@khanacademy/mathjax-renderer";
import "@khanacademy/mathjax-renderer/src/css/mathjax.css";

// if you want \text{} to look good on Safari:
import "@khanacademy/mathjax-renderer/src/css/safari-hacks.css";

// if you want copy-paste support:
import "@khanacademy/mathjax-renderer/src/css/selectable.css";

const renderer = new MathJaxRenderer({
    // shouldFixUnicodeLayout ensures that non-ASCII text is correctly
    // measured and positioned in e.g. \overbrace and \underbrace expressions.
    // Set shouldFixUnicodeLayout to false if you're rendering in an
    // environment without a layout engine, e.g. jsdom.
    shouldFixUnicodeLayout: true,
    fontURL: "https://cdn.kastatic.org/fonts/mathjax",
    // The locale being used to render the output
    locale: "en",
});

const {
    domElement, // the rendered result
    normalizedTex, // TeX with various syntax fixes
    error, // non-null if there was a problem rendering
} = renderer.render("c = \\sqrt{a^2 + b^2}");

// If you're rendering in a browser, be sure to call renderer.updateStyles()
// after each render() to update MathJax's CSS.
renderer.updateStyles();

// If you're rendering in not-a-browser (e.g. during SSR or static site
// generation), call renderer.getStyleSheet() after all expressions on the page
// have been rendered to generate the CSS.
const styleElement = `<style>${renderer.getStyleSheet().textContent}</style>`;

Known Usages

• Webapp
• The Khan Academy mobile app webview
• graphie-to-png (via a Git submodule)
• Perseus in Webapp

Development

One-time setup

It's recommended to clone this repo with ka-clone, which installs Khan Academy's standard git hooks. if you used plain git clone instead, don't worry. You can run ka-clone --repair at any time to install the hooks.

You will need node and yarn for development. If you're a KA engineer, you probably installed these when setting up your computer, but if you don't have them, see this guide.

Versions used for initial development of this repo:

| Program | Recommended Version | | ------- | ------------------- | | node | 16.17.1 | | yarn | 1.22.10 |

Later versions will probably work fine—but if you run into issues, just know that a version mismatch here might be the cause.

Install dependencies and additional git hooks by running:

yarn setup

Everyday Tools

This repo is configured for ergonomic development with Visual Studio Code.

yarn dev          # run the dev UI
yarn test:watch   # run the unit tests on each code change
yarn test:all     # run unit and regression tests (slow!)
yarn tsc          # run the typechecker
yarn tsc -w       # run the typechecker on each code change
yarn lint         # run eslint
yarn verify       # run all checks

Adding a Preprocessing Step

In the preprocessing folder, create a file (ex. to fix a bug) titled something like fix-bug.ts and a second file titled fix-bug.test.ts

Create a function to replace specific TeX (found using regex) with a specific expression (ex below)

export function fixBug(tex: string): string {
    return tex.replace(/<regex>/g, "<new-tex>");

Create unit tests in the test file to confirm the processing step is working. Consider any potential edge cases that your fix might be tested against

import {describe, it, expect} from "@jest/globals";
import {fixBug} from "./fix-bug";

describe("fixBug", () => {
    it("fixes the bug", () => {
        const input = "<old-tex>";
        const fixed = "<new-tex>";
        expect(fixBug(input)).toBe(fixed);
    });
});

Go to the index.ts file and add the fix to the pipeline. It should not be necessary to add to fixNonstandardSyntax as all new math should be MathJax-compatible. As a result, future changes here should be added to workAroundMathjaxDeficiences. (Don’t forget to import the function at the top!)

You should almost always add a regression test (there is a how-to guide for this below too). The only time not to do a regression test is if the feature/fix cannot be covered by these kinds of regression tests (ex. purely visual changes via CSS). To add a regression test, go to the regression-tests folder and create a file titled something like bug-fix.test.ts

Here you will add some standardized code that is used for all the regression tests, but with a couple parts changed (indicated with angle brackets)

import {describe} from "@jest/globals";
import {testEachTexExpressionInFileIsRenderable} from "./test-generator";

describe("the <tex> command in MathJax", () => {
    testEachTexExpressionInFileIsRenderable({
        filename: "<bug-fix>_data.txt",
        separator: "\n\n\n",
    });
});

As you can see, data is needed. Create a file in that same folder titled bug-fix_data.txt. Conduct a regex search of all the content for the specific phrase in question. Add some of that to the file, depending on how many examples there are. The README has more info on this.

Once you add a new regression test, you will want to run all the tests and see if any of the other tests notice a regression. Check to make sure the errors are expected, then update the snapshots by passing -u to the yarn test command. You may be able to update the tests in the IDE interface too.

Summary:

• Create fix file and unit test file in preprocessing
• Add fix to the pipeline in the index file
• Move over to regression-tests and add a test file and data text file to watch for regressions
• Run regression tests and update snapshots (yarn test -u ) Here is an example PR for adding a new preprocessing step

How to Add a New TeX Macro

Head over to the tex-input-config.ts file in src/ts

Follow the format of the other macros there. <Macro-name>: "<new-macro>",

Head over to renderer.test.ts and add in the new macro for testing, following the syntax there

[`\\<Macro-name>`, "<description of how it looks>"]

Run the tests, check for anything unexpected in the results, and then update the snapshots

Here is an example PR for adding a new TeX macro.

Downloading Test Data

It's useful to be able to test MathJax on actual math from our content.

Download a snapshot of English and Hindi content by running:

yarn download-content-snapshot

The download will take about 15 minutes. Once it's done, run

yarn extract-tex

This will scan the downloaded content for TeX expressions and write each expression to a file in data/tex. It takes about 30 minutes.

The download and extract steps are split up because sometimes, the download hangs at 99% (bug in gsutil, maybe?) and has to be manually killed. If you encounter this bug, it's fine to kill the download, then go ahead and run yarn extract-tex without re-running the download.

Adding Regression Tests

If you add features or fix bugs in the renderer, you'll probably want to add regression tests for them. See regression-tests/README.md for details about what the regression tests cover and how they work.

Getting Data For Regression Tests

Assuming you have done all the "Downloading Test Data" steps above, you can search for TeX expressions matching a regex by running:

# NOTE: use single quotes for the regex! Double-quotes in bash use different escaping rules.
yarn search '\\begin\{align(ed)?\}' | pbcopy

The pbcopy command (on macOS only) puts the search results on your clipboard.

Each TeX expression in the results is annotated with the path of the Perseus JSON file it originally came from.

You can then paste the search results into a regression test data file. See the regression-tests folder for examples.

Releasing

The release process uses @changesets/cli.

The workflow is:

• Each feature PR must include a changeset file, generated by running yarn changeset, describing what is in that change and whether a major, minor, or patch version bump is required.
• A GitHub workflow watches the main branch and updates a "Version Packages" PR whenever a feature PR is landed.
• When the "Version Packages" PR is landed, a GitHub workflow updates the package version and changelog based on outstanding changesets, builds a new version of the package, and publishes it to NPM.

Landing the "Version Packages" PR is a manual action (you can use git land for this) but everything else should be automatic.

After releasing, please consider updating the known usages of this repo to the new release.

Upgrading Usages

Webapp

Once a new version is released, the version in webapp will need to be updated. This involves manually updating the version number of MathJax in Webapp's package.json file (services/static/package.json) and then running yarn install in the services/static directory to update the yarn.lock file.

Here is an example PR of bumping the version of MathJax in Webapp: Update mathjax to newest version in webapp (Does not have to be done as an audit as it is here)

It is possible there will be merge conflicts with the package.json. Make sure to confirm the version update is still present after resolving all conflicts!

Graphie Editor

The graphie editor consumes MathJax as a Git submodule. It is connected to a specific commit and does not update automatically, though updating it is similar to updating a regular git repo. Here is an approximate example of how to update it:

cd ~/Khan/graphie-to-png

# go into the submodule directory
cd mathjax-renderer

# download the latest commits from the mathjax-renderer repo
git fetch origin

# update to the desired version
git co vX.Y.Z

# re-pin the submodule to the new version
cd ..
git add mathjax-renderer
git commit

Example PR - Switch the math renderer from KaTeX to MathJax

PR from creating a bundle for the editor. This is regularly done as part of the regular release, so nothing needs to be done regarding the bundle when bumping MathJax in the graphie repo - Build to a single-file bundle, for graphie-to-png

Mobile

This must be done as part of the mobile release cycle. The version is specified in webview/javascript/package.json. Update the version number there, run yarn install, and commit the changes.

Example PR - Bumping the version of MathJax-Render in mobile