prerender-spa-ultra
v1.2.0
Published
Crawls & Pre-renders a site
Downloads
2
Readme
Converts your JavaScript
-powered
SPA
web application into individual *.html
documents (one for each unique URL
your app has) filled with the content that your JavaScript
generated for
them.
Usage — Goals — Limitations & Caveats
(available as github action
, cli
command & npm
package)
What is this for? (purpose)
When your app uses
/deep/link/to/a/page
and you want to get a nice preview of that page when itsURL
is shared.Link sharing over most channels like messaging apps, social networks or other websites will not run
JavaScript
, so they will preview yourURL
based on your static*.html
(which is likely empty). So if you want to preview meaningful things that are shown withJavaScript
like the page main image, or the correct page title you want toprerender
.When you expect slow connection to your SPA or huge size of your
JavaScript
. Instead of making your users wait staring at a blank page you canprerender
that page, so that the html that gets loaded includes the corresponding content and only make the users wait forJavaScript
in order to interact with it
Usage
As a
github action
(» see all GitHub action settings):uses: antitoxic/prerender-spa-ultra@v1 with: website_root: 'path/to/your/spa/dist'
As
CLI
(» see all cli arguments):# will automatically start http server npx prerender-spa-ultra <path/to/your/output/dir>
You can prevent the http server from running or customize further via cli args.
As a
npm
package (» see full config):In contrast to the other approaches, when using
prerender-spa-ultra
programmatically you are responsible for starting the http server for your static files. » See examples how you can do this.import { preRenderSite } from 'prerender-spa-ultra'; //... await preRenderSite({ outputDir: 'path/to/output', // This is the url at which you started the static http server startingUrl: 'http://localhost:8000', });
github action
usage
Below you can find an example of a job definition for prerendering your
repository via a GitHub workflow. Keep in mind that you can skip/remove
actions/setup-node@v3
, if you don't utilize npm
caching.
Whenever a GitHub workflow runs, nodejs
is already available and is the exact
version needed, since prerender-spa-ultra
is written considering this.
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
cache: npm
- name: Install
run: |
npm install
- name: Build
run: |
npm run build
- name: Prereder the freshly built app
uses: @antitoxic/prerender-spa-ultra@v1
id: prerender
with:
website_root: 'path/to/your/app/dist'
# the options below are optional:
max_concurrent_pages: 10
meta_prerender_only: "1"
selector_to_wait_for: "[data-scraper=ready]"
If you are looking to deploy the final prerendered files to Cloudflare you can add the following action in the end of your job:
- name: Deploy to cloudflare
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
run:
npx wrangler pages publish path/to/your/app/dist --project-name
REPLACE_WITH_YOUR_PROJECT_NAME
CLI
usage
Check the help output of npx prerender-spa-ultra
or the self-explanatory
source code of the cli:
https://github.com/antitoxic/prerender-spa-ultra/blob/main/src/prerenderer/cli.ts#L9-L83
npm
package usage
preRenderSite(...)
config type definitions:
https://github.com/antitoxic/prerender-spa-ultra/blob/main/src/prerenderer/prerender.ts#L59-L71
If you are not using the peer dependency serve-handler
to
run http server for your static files you can skip
Debugging problems
You can enable logging:
PRERENDER_SPA_ULTRA_DEBUG=1 npx prerender-spa-ultra ....
# or
PRERENDER_SPA_ULTRA_DEBUG=1 <your command that uses this package programatically>
or as GitHub job step:
- name: Pre-render
uses: antitoxic/prerender-spa-ultra@v1
env:
PRERENDER_SPA_ULTRA_DEBUG: 1
with:
website_root: dist
Serving your SPA static files with local http server
The prerender-spa-ultra
's cli & GitHub action internally use the
serve-handler
npm package to start a http server but there are multiple
alternatives:
cd <path/to/static/files/root/dir/>
# and then one of:
npx serve --single
npx node-static --spa
npx http-server --proxy "http://localhost:8080?"
If you are trying to minimize dependencies and don't want to use those, you can
rely on the included http-server.py
(python 3):
cd <path/to/static/files/root/dir/>
python <path/to/node_modules>/prerender-spa-ultra/src/http-server.py
Limitations & Caveats
- This library shares 2 limitations that any other pre-rendering lib has:
- It doesn't work with hash-based routing of SPAs (i.e.
example.com/#route
). This is because the server never sees the#...
part of the url, so it can't find a file based on it. If you are in this scenario you can try migrating to html5 push history. - Your assets (media files,
*.js
,*.css
) should not be linked as relative paths since pre-rendering creates a nested folder structure to match the urls you have. Instead, those should be linked from your URL root (/...
).
- It doesn't work with hash-based routing of SPAs (i.e.
- The library will ignore query params and consider all urls having matching
pathname
to be one and the same url. You can override this by providing a customcleanUrl
andgetFilename
functions topreRenderSite
. It will be up to you to configure your http server to route such urls with query params to their respective static file (created bygetFilename
). - The default
cleanUrl
trims any slashes, which meanssome/url/path/
andsome/url/path
will be considered the same
Goals of prerender-spa-ultra
- Simplest prerender out there
- Single programmatic dependency (
puppeteer-core
), nothing else besides the peer dependency for the automatic http server in CLI (which you can opt out). This also means saving build-time otherwise spend downloading packages & binaries on every build - Pre-renderer with the lowest bug surface (including dependencies) - written in the most concise way possible while keeping readability-first design
- Know what you are executing — One of the goals for
prerender-spa-ultra
is to be easy to understand from just reading the code. - Uses already available packages & binaries on the OS it's running on (see Built with CI/CD & JAMSTACK in mind)
- Single programmatic dependency (
- Crawls your site. It will find all you URLs that need prerendering as long as they are linked from another page. You don't need to provide explicit list of urls to prerender, just pass the URL you want to start with.
- Optimized for speed of pre-rendering
- Stops crawling as soon as possible
- Blocks unnecessary for the pre-rendering resources from loading (blocking
is configurable). By default, it blocks the following:
- fonts
- images & media
- some known third party scripts (your site shouldn't fail without them)
- Concurrent crawling & concurrent pre-rendering (concurrency is
configurable)
- Creates a pool of browser pages for pre-rendering in parallel
- Reuses the pages instead of destroying and recreating them
- All IO operations are async, no filesystem or network calls that are blocking
- Built with CI/CD & JAMSTACK in mind
- Provides configurations for Github Action & Cloudflare pages deployment
- Targets and uses already available packages on those build images
Non-goals of prerender-spa-ultra
prerender-spa-ultra
has a narrow goal. Let's keep expectation clear:
prerender-spa-ultra
is not going to optimize the output html files. It's far more efficient to do that beforehand in your main SPA assets-build step.prerender-spa-ultra
is not going to prerender any site you provide. This is specifically made to work together with the local http static file server included in it. The purpose of this is making sure you get the maximum performance + taking care of some edge case scenarios in headless chromeprerender-spa-ultra
is not going to provide 100% of the options as CLI arguments. Only the basic customizations are possible by passing CLI arguments. If you want to do something more advanced, importprerender-spa-ultra
as a nodejs module and call it with all the options you need.prerender-spa-ultra
is not going to install chrome or chromium. If you are using GitHub workflows or similar, it's likely to have it already installed. Otherwise, you can use the OS package manager to install or usenode-chromium
(npm install chromium
)
Funding
Will be opened very soon, waiting for GitHub sponsorship approval :)
Prior art
- https://github.com/egoist/presite
- https://github.com/stereobooster/react-snap
- https://github.com/dattran92/site-prerender
- https://github.com/JoshTheDerf/prerenderer
- https://github.com/sPavl0v/react-spa-prenderer
- https://github.com/chrisvfritz/prerender-spa-plugin
External Pre-render Services available
- https://webprerender.io/pricing/
- https://prerender.io/
Marketing locations
- https://github.com/automata/awesome-jamstack
- (old) https://www.tnd.dev/tool/
Background & References
- Not ideal to use
JSDOM
since it can't safely execute<script/>
s - Good Jamstack intro video: https://vimeo.com/163522126 and the rest: https://jamstack.org/resources/videos/
- Available binary packages in various CI/CD pre-build images:
- https://github.com/cloudflare/pages-build-image/discussions/1 & https://developers.cloudflare.com/pages/platform/build-configuration/
- https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md
- https://github.com/netlify/build-image/blob/focal/included_software.md
- Reserved tlds for local work https://news.ycombinator.com/item?id=12578908
- GitHub actions running locally:
- https://stackoverflow.com/questions/59241249/how-to-run-prerender-spa-ultra-github-actions-workflows-locally
- https://github.com/nektos/act
Cloudflare integration
https://developers.cloudflare.com/workers/wrangler/ci-cd/
To use cloudflare cli (wrangler
) from CI like GitHub actions you need to
create CLOUDFLARE_API_TOKEN
and add it as a secret in that CI environment
Maybe you will need to set CLOUDFLARE_ACCOUNT_ID
if you have more than 1
account associated with this API token.
Must expose GitHub secret as ENV variable (not done by default)