Next.js Web Scraper Playground

Build and test your own web scraper APIs with Next.js API Routes and cheerio.

For more info on building your own scraping APIs, checkout Create a public API by web scraping in NextJS by Michael Liendo and Build a web scraper with Node by Ayooluwa Isaiah.

To contribute to or make your own version of this project, grab the code from the repository and install dependencies:

npm install

Run the app in the development mode:

npm run dev

Open http://localhost:3000 to view in the browser.

Apply auto-formatting as you make changes:

npm run format-watch

This project includes Cypress integration tests. Run tests in a browser:

npm run cypress

Run headless tests:

npm run test

To run tests then push to main:

npm run deploy

Deploy to Vercel

Vercel offers a zero-configuration single-command deployment.

1. Install vercel from vercel.com/cli or via node with npm install -g vercel.

2. Run vercel from your project directory and follow the prompts.

Read more about the Vercel CLI here. You can also set up automatic deployments from git branches.

This project was bootstrapped with Create Next App.

Find the most recent version of this guide at here. And check out Next.js repo for the most up-to-date info.