wordpress-to-github

A module for taking content from Wordpress instances and pushing the HTML content and JSON meta to Github.

How does it work?

The service scans a list of WordPress projects. Each project's WordPress site is queried through the WordPress API. The service compares the objects from the API to the project's target GitHub branch. Content changes are recorded in GitHub as commits.

Caching

The service will ask WordPress for object counts and last updated timestamps (and cache them) to determine if it should run. If you are working on the WordPress API output without updating the content itself, changes may not appear until the cache is reset.

Config files

There are a few configuration files that need to be used.

wordpress-to-github.config.json

Controls how the service will place content in GitHub. This file belongs in your target repo (such as drought.ca.gov).

{
  "$schema": "https://raw.githubusercontent.com/cagov/wordpress-to-github/main/wordpress-to-github/schemas/wordpress-to-github.config.schema.json",
  "meta": {
    "title": "wordpress-to-github endpoints config file",
    "description": "wordpress-to-github endpoints config file"
  },
  "data": {
    "disabled": false,
    "PostPath": "wordpress/posts",
    "PagePath": "wordpress/pages",
    "MediaPath": "wordpress/media",
    "ApiRequests": [
      {
        "Destination": "wordpress/menus/header-menu.json",
        "Source": "/wp-json/menus/v1/menus/header-menu",
        "ExcludeProperties": ["description"]
      }
    ],
    "GeneralFilePath": "wordpress/general/general.json",
    "ExcludeProperties": ["content", "_links"],
    "HideAuthorName": false
  }
}

| Name | Description | | :---------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------ | | disabled | Set to true to disable processing for this project. | | PostPath | Where should the posts go? | | PagePath | Where should the pages go? | | MediaPath | Where should image media go? | | ApiRequests | A collection of API requests to write to the repo. | | ApiRequests.Destination | The output path (in the repo) for an API request. | | ApiRequests.Source | The WordPress API source. This should be an absolute path against the top-level domain of your WordPress site, likely beginning with "/wp-json/". | | ApiRequests.ExcludeProperties | A collection of property keys to remove from the output. | | GeneralFilePath | The full path and filename for a general.json file that contains information about the whole site. | | ExcludeProperties | Which WordPress properties should we suppress in output? | | HideAuthorName | true to show user number for Author output instead of Name |

Sample output

These are examples of the types of files that will be written to the target GitHub repo by this service.

general / general.json

This file will contain global information for the WordPress istance

{
  "meta": {
    "api_version": "v2",
    "api_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json/wp/v2/",
    "object_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json?_fields=description,gmt_offset,name,namespaces,timezone_string,home,url",
    "process": {
      "source_code": "https://github.com/cagov/wordpress-to-github",
      "source_data": "https://as-go-covid19-d-001.azurewebsites.net",
      "deployment_target": "https://github.com/cagov/automation-development-target/tree/main"
    },
    "refresh_frequency": "as needed"
  },
  "data": {
    "name": "COVID-19",
    "description": "Content site for COVID-19",
    "url": "http://as-go-covid19-d-001.azurewebsites.net",
    "home": "https://as-go-covid19-d-001.azurewebsites.net",
    "gmt_offset": "0",
    "timezone_string": "",
    "namespaces": ["oembed/1.0", "wp/v2", "wp-site-health/v1"]
  }
}

posts / [slug].json

All posts .son files will appear next to their .html content files.

{
  "meta": {
    "created_date": "2020-04-13T19:51:57",
    "updated_date": "2020-07-14T22:32:38",
    "field_reference": "https://developer.wordpress.org/rest-api/reference/posts/",
    "api_version": "v2",
    "api_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json/wp/v2/",
    "object_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json/wp/v2/posts/1196",
    "process": {
      "source_code": "https://github.com/cagov/wordpress-to-github",
      "source_data": "https://as-go-covid19-d-001.azurewebsites.net",
      "deployment_target": "https://github.com/cagov/automation-development-target/tree/main"
    },
    "refresh_frequency": "as needed"
  },
  "data": {
    "id": 1196,
    "date": "2020-04-13T19:51:57",
    "date_gmt": "2020-04-13T19:51:57",
    "guid": "http://as-go-covid19-d-001.azurewebsites.net/?p=1196",
    "modified": "2020-07-14T22:32:38",
    "modified_gmt": "2020-07-14T22:32:38",
    "slug": "404",
    "status": "publish",
    "type": "post",
    "link": "https://as-go-covid19-d-001.azurewebsites.net/2020/04/13/404/",
    "title": "404",
    "excerpt": "<p>We couldn&#8217;t find this page. Want to search? No pudimos encontrar esta página. Chúng tôi không thể tìm thấy trang này. لم نتمكن من العثور على هذه الصفحة. 이 페이지를 찾을 수 없습니다. 我们找不到此页面。</p>\n",
    "author": "Carter Medlin",
    "featured_media": 0,
    "comment_status": "closed",
    "ping_status": "open",
    "sticky": false,
    "template": "",
    "format": "standard",
    "meta": [],
    "categories": ["error page"],
    "tags": ["do-not-crawl", "machine-translated"],
    "wordpress_url": "https://as-go-covid19-d-001.azurewebsites.net/2020/04/13/404/"
  }
}

pages / [slug].json

Pages work like posts, with some differences.

{
  "meta": {
    "created_date": "2021-02-03T21:44:55",
    "updated_date": "2021-02-03T21:44:55",
    "field_reference": "https://developer.wordpress.org/rest-api/reference/pages/",
    "api_version": "v2",
    "api_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json/wp/v2/",
    "object_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json/wp/v2/pages/8932",
    "process": {
      "source_code": "https://github.com/cagov/wordpress-to-github",
      "source_data": "https://as-go-covid19-d-001.azurewebsites.net",
      "deployment_target": "https://github.com/cagov/automation-development-target/tree/main"
    },
    "refresh_frequency": "as needed"
  },
  "data": {
    "id": 8932,
    "date": "2021-02-03T21:44:55",
    "date_gmt": "2021-02-03T21:44:55",
    "guid": "http://as-go-covid19-d-001.azurewebsites.net/?page_id=8932",
    "modified": "2021-02-03T21:44:55",
    "modified_gmt": "2021-02-03T21:44:55",
    "slug": "home",
    "status": "publish",
    "type": "page",
    "link": "https://as-go-covid19-d-001.azurewebsites.net/",
    "title": "Home",
    "excerpt": "<p>homepage</p>\n",
    "author": "Aaron Hans",
    "featured_media": 0,
    "parent": 0,
    "menu_order": 0,
    "comment_status": "closed",
    "ping_status": "closed",
    "template": "",
    "meta": [],
    "wordpress_url": "https://as-go-covid19-d-001.azurewebsites.net/"
  }
}

media / [year] / [month] / [slug].json

Media files include sizes.

{
  "meta": {
    "created_date": "2020-07-14T18:55:54",
    "updated_date": "2020-07-14T18:55:54",
    "field_reference": "https://developer.wordpress.org/rest-api/reference/pages/",
    "api_version": "v2",
    "api_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json/wp/v2/",
    "object_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-json/wp/v2/media/4826",
    "process": {
      "source_code": "https://github.com/cagov/wordpress-to-github",
      "source_data": "https://as-go-covid19-d-001.azurewebsites.net",
      "deployment_target": "https://github.com/cagov/automation-development-target/tree/main"
    },
    "refresh_frequency": "as needed"
  },
  "data": {
    "id": 4826,
    "date": "2020-07-14T18:55:54",
    "date_gmt": "2020-07-14T18:55:54",
    "guid": "/wp-content/uploads/2020/07/15485_lores.jpg",
    "modified": "2020-07-14T18:55:54",
    "modified_gmt": "2020-07-14T18:55:54",
    "slug": "15485_lores",
    "status": "inherit",
    "type": "attachment",
    "link": "https://as-go-covid19-d-001.azurewebsites.net/2020/05/11/masks-and-ppe/15485_lores/",
    "title": "15485_lores",
    "author": "Aaron Hans",
    "comment_status": "closed",
    "ping_status": "closed",
    "template": "",
    "meta": [],
    "description": "<p class=\"attachment\"><a href='/wp-content/uploads/2020/07/15485_lores.jpg'><img width=\"300\" height=\"200\" src=\"/wp-content/uploads/2020/07/15485_lores-300x200.jpg\" class=\"attachment-medium size-medium\" alt=\"\" loading=\"lazy\" srcset=\"/wp-content/uploads/2020/07/15485_lores-300x200.jpg 300w, /wp-content/uploads/2020/07/15485_lores.jpg 700w\" sizes=\"(max-width: 300px) 100vw, 300px\" /></a></p>\n",
    "caption": "",
    "alt_text": "",
    "media_type": "image",
    "mime_type": "image/jpeg",
    "media_details": {
      "width": 700,
      "height": 466,
      "file": "2020/07/15485_lores.jpg",
      "sizes": {
        "medium": {
          "file": "15485_lores-300x200.jpg",
          "width": 300,
          "height": 200,
          "mime_type": "image/jpeg",
          "source_url": "/wp-content/uploads/2020/07/15485_lores-300x200.jpg"
        },
        "thumbnail": {
          "file": "15485_lores-150x150.jpg",
          "width": 150,
          "height": 150,
          "mime_type": "image/jpeg",
          "source_url": "/wp-content/uploads/2020/07/15485_lores-150x150.jpg"
        },
        "full": {
          "file": "15485_lores.jpg",
          "width": 700,
          "height": 466,
          "mime_type": "image/jpeg",
          "source_url": "/wp-content/uploads/2020/07/15485_lores.jpg"
        }
      },
      "image_meta": {
        "aperture": "0",
        "credit": "",
        "camera": "",
        "caption": "",
        "created_timestamp": "0",
        "copyright": "",
        "focal_length": "0",
        "iso": "0",
        "shutter_speed": "0",
        "title": "",
        "orientation": "0",
        "keywords": []
      }
    },
    "post": 2997,
    "source_url": "/wp-content/uploads/2020/07/15485_lores.jpg",
    "wordpress_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-content/uploads/2020/07/15485_lores.jpg",
    "sizes": [
      {
        "type": "full",
        "path": "2020/07/15485_lores.jpg",
        "wordpress_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-content/uploads/2020/07/15485_lores.jpg",
        "file": "15485_lores.jpg",
        "width": 700,
        "height": 466,
        "mime_type": "image/jpeg",
        "source_url": "/wp-content/uploads/2020/07/15485_lores.jpg"
      },
      {
        "type": "medium",
        "path": "2020/07/15485_lores-300x200.jpg",
        "wordpress_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-content/uploads/2020/07/15485_lores-300x200.jpg",
        "file": "15485_lores-300x200.jpg",
        "width": 300,
        "height": 200,
        "mime_type": "image/jpeg",
        "source_url": "/wp-content/uploads/2020/07/15485_lores-300x200.jpg"
      },
      {
        "type": "thumbnail",
        "path": "2020/07/15485_lores-150x150.jpg",
        "wordpress_url": "https://as-go-covid19-d-001.azurewebsites.net/wp-content/uploads/2020/07/15485_lores-150x150.jpg",
        "file": "15485_lores-150x150.jpg",
        "width": 150,
        "height": 150,
        "mime_type": "image/jpeg",
        "source_url": "/wp-content/uploads/2020/07/15485_lores-150x150.jpg"
      }
    ],
    "path": "2020/07/15485_lores.jpg"
  }
}