Convertiv Cloud Development Kit

This is a set of packages built on top of the AWS CDK that deploy specific resource stacks for common convertiv applications. This application is written in typescript on top of AWS's CDK typescript toolkit. Its designed to codify infrastructural best practices, and make a deployable infrastructure artifact.

This package is deployed to NPM as convertiv-cdk

Never include secrets, variables or customer data in this project

All client customization should occur in a client specific usage of this library. This library should be abstract and reusable across clients.

CDK compiles down to CloudFormation YML, so familiarity with that will be handy.

Requirements

• Node
• NPM
• AWS CLI
• AWS CDK CLI (npm install -g aws-cdk)

Publishing

This project is published to npm as convertiv-cdk. When adding new code to the repository, first increment the package number in package.json and then run npm run build to compile the code as js.

Then run npm publish to publish the package to the npm repository. Use SemVer major.minor.patch and all minor changes should be backwards compatible.

Structure

There are three main parts of this project packages, stacks, and the index.ts.

Index.ts

Index.ts allows you to include

Packages

Packages are the smallest component part. They are designed to be fully agnostic, single purpose building blocks. For example, we have a networking package that implements a simple VPC and the networking peering to deploy safe infrastructure.

Use packages when you want to group together several CDK elements for a particular task.

Each package should provide a settings interface so developers can see what settings are accepted by the package. Packages should not have any dependencies on each other. If you want to share a resource between packages, the first package should attach the resource to a property, and the second package should accept the resource as a typed setting in the package settings interface. This way the packages are agnostic.

For example, say your first package creates a security group that you will need to modify in a later package. In your first package, define a property of the class public securityGroup: ISecurityGroup Then in your pacage you can do this.securityGroup = new SecurityGroup().

Your second package should define a settings interface

export interface SecondPackageSettings {
  securityGroup: SecurityGroup;
}

Then in your stack you can do something like this

  const firstPackage = new FirstPackage(app, 'first-package', { ...settings });
  const secondPackage = new SecondPackage(app, 'second-package', { securityGroup: firstPackage.securityGroup })

This way second package could be used fully agnostic of the stack, and accept any security group. For instance, instead of passing a security group from FirstPackage, you could declare an ad hoc security group or fetch the security group from the WebService package.

Current packages

The current packages this provides are -

• BaseModule - This is an abstract package that every package inherits to provide simple validation and structure. It does nothing on its own.
• Cluster - This creates a ECS cluster that services can be deployed to
• Database - This creates a database instance, along with a security group, a subnet group, and basic ingress rules
• Network - This creates the basic VPC and adds the subnets and peering rules to manage network access between resources.
• __QueueServicev - This creates a queue fargate task to run background tasks.
• WebService - This provisions a Fargate Loadbalenced service along with the tasks, security group, and deploy user needed to manage the data.
• WebCommon - This provisions a set of requirements of many web apps and web sites. It creates a public and private s3 button, a service user that can send emails as well as read and write those buckets, a file access user that has read access to those buckets, and the correct policies for those users.

Stacks

Stacks are composites of packages and cdk structures for specific tasks. A stack accepts a settings object and compiles together a set of packages. The stack should provide an interface for settings so developers know what can be passed to them.

Stacks extend the CDK stack object, so it is not possible to share data between stacks. Stacks should be fully encapsulated, accepting a settings object but expected to run independently.

A stack should orchestrate everything you need, as much as possible. There are some cases where your project may have multiple stacks in it, but stacks should implement feature complete business requirements as much as possible.

For instance, a stack might implement a S3 Cloudfront web site implementation. Do not write one stack to deploy the s3 resource and one stack to deploy the cloud front. If you want encapsulation of those functions use packages.

Multistacks

There are two known examples of places where you might use multiple stacks in your application

• Separating environments
  • Its very handy to boot staging and production separately. In the above s3/cloudfront example, you may have one stack, but in your project boot two instances of the stack, staging and prod, with seperate users, buckets and cf distros.
• Multistage
  • Some times, the stack needs to be split into two stages. The initial stage gets deployed and then you deploy a second stage over top. This is mostly because of unreconcilable conflicts. In general this pattern should be avoided

Current Stacks

• WebService - This deploys the convertiv app development infrastructure. It provisions a database, a ECS cluster, services for the web task and a queue task, deploy users, and the required glue code
• ETLPipeline - This is a work in progress stack designed to support the convertiv ETL pipelines.
  Currently it provisions just an ECR repo and users for deploying and accessing that ecr repo.

The cdk.json file tells the CDK Toolkit how to execute your app.

Useful commands

• npm run build compile typescript to js
• npm run watch watch for changes and compile
• npm run test perform the jest unit tests
• cdk deploy deploy this stack to your default AWS account/region
• cdk diff compare deployed stack with current state
• cdk synth emits the synthesized CloudFormation template