@newskit-render/feed

A package used for the generation of different types of sitemaps or Rss feed. You can see an example here: @newskit-render/core

Getting started

Add it as a dependency:

npm install @newskit-render/feed

or

yarn add @newskit-render/feed

Create sitemap pages

Under your /pages/api folder create sitemap.ts, pages-sitemap.ts, competitions-sitemap.ts and news-sitemap.ts files.

update the next.config.js file to add rewrites:

async rewrites() {
  return [
    {
      source: '/sitemap.xml',
      destination: '/api/sitemap',
    },
    {
      source: '/pages-sitemap.xml',
      destination: '/api/pages-sitemap',
    },
    {
      source: '/competitions-sitemap.xml',
      destination: '/api/competitions-sitemap',
    },
    {
      source: '/news-sitemap.xml',
      destination: '/api/news-sitemap',
    },
  ]
},

Create Rss page

Under your /pages/api folder create feed.ts file

update the next.config.js file to add rewrites:

async rewrites() {
  return [
    {
      source: '/feed',
      destination: '/api/feed',
    },
  ]
},

see rssFeed funtion

Implementation

genericSitemap

genericSitemap - this function will handle the logic for 4 different views that correspond to the sitemap endpoints you should already have set up (see Create sitemap pages section).

• sitemap.xml will show a sitemapindex of sitemaps by date.
• sitemap.xml?query=1 will show all articles created on that day, example sitemap.xml?yyyy=2022&mm=10&dd=14.
• pages-sitemap.xml will show all pages including homepage - custom pages not available in the api can be added.
• competitions-sitemap.xml will show all competitions.

genericSitemap is passes a context object, depending on the view, it will include the following parameters:

dataType can be articles, pages or competitions. Should be set to show pages or competitions view. Not setting this will result in default of articles used by sitemap.xml and sitemap.xml?query=1.

res (required) from the Next.js context object.

query from the Next.js context object - is required for sitemap.xml?query=1 view.

publisher (required) is the name of the publisher in the NewsKit API, this string will be used for fetching the articles for the correct publication.

publicationName (required) is the name of the publication, used inside the sitemap <news:name>${publicationName}</news:name> tag.

domain (required) is the domain of your website.

firstArticleDate is the date of the first article you have published. This will determine from when the sitemapindex of sitemaps by date starts from. If not added you will get the last 30 days.

CustomStaticPageCollection provides the possibility to manually add a list of static pages, that do not exist in an API. This will work with pages and competitions, but it would be unlikely to be used with competitions. The format of the static page is in the example pages-sitemap.xml below, but has 2 properties. slug (required) and url

changefreq you can change the default changefreq for entries on the sitemap

priority you can change the default priority for entires on the sitemap

Example for sitemap.xml and sitemap.xml?query=1

import { NextApiRequest, NextApiResponse } from 'next'
import { genericSitemap } from '@newskit-render/sitemap'
import { Publisher } from '@newskit-render/api'
import { getHost } from '@newskit-render/shared-components'

const handler = async (req: NextApiRequest, res: NextApiResponse) =>
  genericSitemap({
    res,
    query: req.query,
    publisher: process.env.PUBLISHER as Publisher,
    domain: new URL(getHost(req)).host,
    firstArticleDate: process.env.SITEMAP_FIRST_PUBLICATION_DATE,
    publicationName: process.env.SITEMAP_PUBLICATION_NAME,
  })

export default handler

You will find the env vars in /helm/value-{env}.yaml

Example values are:

publisher: 'VIRGIN',
domain: 'virginradio.co.uk',
firstArticleDate: '2016-3-1',
publicationName: 'Virgin Radio',

Example for pages-sitemap.xml

import { NextApiRequest, NextApiResponse } from 'next'
import { genericSitemap, CustomStaticPage } from '@newskit-render/feed'
import { Publisher } from '@newskit-render/api'
import { getHost } from '@newskit-render/shared-components'

const defaultCustomStaticPagesCollection: CustomStaticPage[] = [
  {
    slug: 'test-custom-page',
  },
]
const handler = async (req: NextApiRequest, res: NextApiResponse) =>
  genericSitemap({
    res,
    dataType: 'pageList',
    publisher: process.env.PUBLISHER as Publisher,
    domain: new URL(getHost(req) as string).host,
    publicationName: process.env.SITEMAP_PUBLICATION_NAME,
    customStaticPageCollection: defaultCustomStaticPagesCollection,
    changefreq: 'hourly',
    priority: '0.8',
  })

export default handler

Example of competitions-sitemap.xml

import { NextApiRequest, NextApiResponse } from 'next'
import { genericSitemap } from '@newskit-render/feed'
import { Publisher } from '@newskit-render/api'
import { getHost } from '@newskit-render/shared-components'

const handler = async (req: NextApiRequest, res: NextApiResponse) =>
  genericSitemap({
    res,
    dataType: 'competitionList',
    publisher: process.env.PUBLISHER as Publisher,
    domain: new URL(getHost(req) as string).host,
    publicationName: process.env.SITEMAP_PUBLICATION_NAME,
  })

export default handler

newsSitemap

This function will handle the logic for news-sitemap.xml and creates the sitemap required by Google News. You will need to register the sitemap with Google.

newsSitemap - will receive the following parameters:

context (required) - Next.js context object (only needs res from context but its easier to destructor as below)

publisher, domain, publicationName (required): Required properties for correctly building the sitemap.xml. The domain is the domain of your website. publisher is the name of the publisher in the NewsKit API, this string will be used for fetching the articles for the correct publication. publicationName is the name of the publication, used inside the sitemap <news:name>${publicationName}</news:name> tag.

changefreq you can change the default changefreq for entries on the sitemap

priority you can change the default priority for entires on the sitemap

Below an example.

import { NextApiRequest, NextApiResponse } from 'next'
import { newsSitemap, PublisherGroup } from '@newskit-render/sitemap'

const handler = async (req: NextApiRequest, res: NextApiResponse) =>
  newsSitemap({
    res,
    publisher: process.env.PUBLISHER as PublisherGroup,
    domain: new URL(getHost(req)).host,
    publicationName: process.env.SITEMAP_PUBLICATION_NAME,
  })

export default handler

robots.txt

Your sitemap files will need to be added to your robots.txt file something like:

User-agent: *
Disallow: /search/
Disallow: /*?s=*
Disallow: *&s=*
Sitemap: https://example.co.uk/sitemap.xml
Sitemap: https://example.co.uk/news-sitemap.xml
Sitemap: https://example.co.uk/pages-sitemap.xml
Sitemap: https://example.co.uk/competitions-sitemap.xml

This can be done by adding it into the next.js public folder or you can create something more dynamic to take into account all your environments:

Under your /pages/api folder create robots.ts file with content like:

import { NextApiRequest, NextApiResponse } from 'next'
import { getHost } from '@newskit-render/shared-components'

export default function handler(req: NextApiRequest, res: NextApiResponse) {
  const crawable =
    process.env.NODE_ENV === 'production' ? 'Allow: /' : 'Disallow: /'
  const robots = `User-agent: *
${crawable}
Sitemap: ${new URL(getHost(req) as string)}sitemap.xml
Sitemap: ${new URL(getHost(req) as string)}news-sitemap.xml
Sitemap: ${new URL(getHost(req) as string)}pages-sitemap.xml
Sitemap: ${new URL(getHost(req) as string)}competitions-sitemap.xml`

  res.statusCode = 200
  res.setHeader('Content-Type', 'text/plain')
  res.write(robots)
  res.end()
}

update the next.config.js file to add rewrites:

async rewrites() {
  return [
    {
      source: '/robots.txt',
      destination: '/api/robots',
    },
  ]
},

rssFeed

rssFeed - this funtion will handle the logic for feed.ts route. It will receive the following parameters:

context (required) - Next.js context object (only needs res from context but its easier to destructor as below)

titeAttributes object with the following attributes:

title - string - title of the rss feed link - string - link of rss feed (example - https://talksport.com/feed/) description - string - describe what the feed is for lastBuildDate - string - date in RFC-822 date-time - denotes last time content of feed was updated language - optional - string - language code of feed - default: en-US updatePeriod - optional - string - indicates how often feed updates so feed users know how often they should pole it. Can be 'hourly' | 'daily' | 'weekly' | 'monthly' | 'yearly' - default: hourly updateFrequency - optional - number - representing the number of times the feed should be refreshed during the updatePeriod - default: 1. logoUrl - string - url to site icon imageSourceSizes - array of string values - images in the <content:endcoded> sections use the <picture> tag with <source> tag. Default widths are provided for these but you can override them with an array ['100', '200', '300', '400', '500']. Only add 5 values to the array.

example

import { NextApiRequest, NextApiResponse } from 'next'
import { rssFeed, UpdatePeriod } from '@newskit-render/feed'

const handler = async (req: NextApiRequest, res: NextApiResponse) =>
  rssFeed({
    res,
    titeAttributes: {
      title: 'Demo Site',
      link: `/feed`,
      description: 'Newskit Render Demo site',
      lastBuildDate: new Date().toUTCString(),
      language: 'en-US',
      updatePeriod: 'hourly' as UpdatePeriod,
      updateFrequency: 1,
      logoUrl: `/favicon.ico`,
    },
  })

export default handler