amplify-sequential-table-create

How to use to create a new Amplify backend environment

Install the plugin.

# Install the npm package globally:
npm i -g amplify-sequential-table-create

# Tell Amplify CLI to scan for plugins - it should automatically find this one due to naming / install location defaults
amplify plugin scan

Now, each time you want to create a new backend environment:

# first save, commit, and push all changes, because some local files will get messed with and you don't want to have to untangle that

# you may need to "sudo" this command - it likely won't error until the end if so, and then you'll have a somewhat annoying recovery step to fix up your team-provider-info.json file
amplify env add

? Do you want to use an existing environment? (Y/n)
# No - type "n"

? Enter a name for the environment (dev)
# enter name of your choosing - up to 10 lowercase alpha characters

? Do you want to use an AWS profile (Y/n)
# Yes - press enter to accept default Y

? Please choose the profile you want to use (Use arrow keys)
# you should have at least one profile set up; make sure you select one that will point to the AWS Organization where you are intending to create this environment

# Now, wait for a few things to be created (this is very quick, a few minutes max)

# now for each lambda layer, you will be asked:
? Choose the environment to import the layer access settings from:
# choose "Apply default access (Only this AWS account)" for each of them

This process likely made changes to your local files. git checkout -- . to discard them.

Next, we need to pre-generate the CloudFormation templates that will be used or this plugin will not work and you won't actually be able to initialize your environment:

amplify api gql-compile

Finally, push up the changes. This will tie up your terminal for about a half hour.

amplify push --no-gql-override --y

You should see a bunch of log messages beginning with "Plugin sequential-table-create" telling you that dependencies have been added.

Why would one need this plugin?

The @model directive from the AWS Amplify framework causes a Dynamo table to be created. All of these tables are created in parallel. Dynamo has limits on how many tables can be created in parallel; if an amplify project has more than some number of @models in its schema, creating a new environment will fail with an error:

Subscriber limit exceeded: You have exceeded the maximum number of indexed tables that can be created simultaneously (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: LimitExceededException;

One solution to this issue is to create the tables serially instead of in parallel. To do this, we can insert an entry in the DependsOn list of some dynamo resources in the CloudFormation template, such that they depend on other dynamo resources and therefore will not start creating until their dependencies have finished.

Because the CloudFormation template that creates the Dynamo tables is generated by Amplify upon each build, and is not in source control, making changes to it manually will still not allow the cloud CD pipeline to function; it will simply get re-created from scratch in there, and any local changes you've made will be lost. Thus you must make the changes locally, and then do an amplify push --no-gql-override command so that your CLI will actually push up the edited files instead of regenerating them again.

This package is a plugin to the amplify CLI that will automatically make those DependsOn edits for you upon calling amplify push, freeing you from needing to make tedious local copy+paste changes. At the moment, you must still generate the built cloudformation template prior to pushing and then include the --no-gql-override flag in the push - I haven't figured out Amplify's plugin framework sufficiently to avoid this, though it sure seems like it's not supposed to be necessary.

Acknowledgements

Great thanks to @mormsbee from whom we forked this project.