aws-data-landing-zone
v0.0.44
Published
AWS CDK Data Landing Zone construct
Downloads
660
Readme
Data Landing Zone
- Getting Started
- Key Features
- Intended Audience
- Core Principles
- Integrated AWS Services
- How it works
- Sponsors
- Contributing
- Docs
- Roadmap
The Data Landing Zone (DLZ) is a CDK construct designed to accelerate AI and data-related projects. It provides an opinionated Landing Zone, laying the foundation for a multi-account AWS strategy, so you can focus on delivering data and AI solutions.
The DLZ can be deployed in existing AWS Organizations or used in greenfield projects. It supports setups ranging from small organizations with a few accounts to large enterprises with hundreds of accounts.
See the Documentation Website: https://datalandingzone.com/ for more information.
The CDK construct is available in both TypeScript and Python. Example GitHub repositories showing usage:
Getting Started
Typescript
Install the aws-data-landing-zone CDK construct from NPM:
npm install aws-data-landing-zoneUse the construct as a stack in your CDK application. Replace the organizationId, ouId, ous, accountId, and
regions with your own values as per AWS Control Tower & AWS Organizations.
The example below shows a simple DLZ setup with two accounts, one for development and one for production, creating two non-overlapping VPCs in two regions in each account.
import {App} from 'aws-cdk-lib';
import { DataLandingZone, Defaults } from 'aws-data-landing-zone';
const app = new App();
const dlz = new DataLandingZone(app, {
...
regions: {
global: Region.EU_WEST_1,
regional: [Region.US_EAST_1],
},
budgets: [
...Defaults.budgets(100, 20, {
slack: slackBudgetNotifications,
emails: ['[email protected]'],
}),
],
denyServiceList: [
...Defaults.denyServiceList(),
'ecs:*',
],
organization: {
organizationId: 'o-0f5h921gk9',
root: { accounts: { management: { accountId: '123456789012', }, }, },
ous: {
workloads: {
ouId: 'ou-h2l0-gjr36ikn',
accounts: [{
name: 'development',
accountId: '123456789012',
type: DlzAccountType.DEVELOP,
vpcs: [
Defaults.vpcClassB3Private3Public(0, Region.EU_WEST_1), // CIDR 10.0.0./19
Defaults.vpcClassB3Private3Public(1, Region.US_EAST_1), // CIDR 10.1.0./19
]
},{
name: 'production',
accountId: '123456789012',
type: DlzAccountType.PRODUCTION,
...
},
...AS MANY ACCOUNTS AS DESIRED...
]
},
},
...
},
network: {
nats: [
{
name: "development-eu-west-1-internet-access",
location: new NetworkAddress('development', Region.EU_WEST_1, 'default', 'public', 'public-1'),
allowAccessFrom: [
new NetworkAddress('development', Region.EU_WEST_1, 'default', 'private')
],
type: {
gateway: {
eip: ... //Optional
}
}
},
],
bastionHosts: [
{
name: 'default',
location: new NetworkAddress('development', Region.EU_WEST_1, 'default', 'private', 'private-1'),
instanceType: InstanceType.of(InstanceClass.T3, InstanceSize.MICRO),
}
]
}
});Continue reading Getting Started on the DLZ documentation site.
Python
Install the aws-data-landing-zone CDK construct from PyPi:
pip install aws-data-landing-zoneUse the construct as a stack in your CDK application. Replace the organizationId, ouId, ous, accountId, and
regions with your own values as per AWS Control Tower & AWS Organizations.
The example below shows a simple DLZ setup with two accounts, one for development and one for production, creating two non-overlapping VPCs in two regions in each account.
import aws_cdk as cdk
import aws_data_landing_zone as dlz
app = cdk.App()
dlz.DataLandingZone(app,
...
regions=dlz.DlzRegions(
global_=dlz.Region.EU_WEST_1,
regional=[dlz.Region.US_EAST_1],
),
budgets=[
*dlz.Defaults.budgets(
100,
20,
slack=slack_budget_notifications,
emails=["[email protected]"],
),
],
deny_service_list=[
*dlz.Defaults.deny_service_list(),
"ecs:*"
],
organization=dlz.DLzOrganization(
organization_id='o-0f5h921gk9',
root=dlz.RootOptions(
accounts=dlz.OrgRootAccounts(
management=dlz.DLzManagementAccount(account_id='123456789012'),
),
),
ous=dlz.OrgOus(
workloads=dlz.OrgOuWorkloads(
ou_id='ou-h2l0-gjr36ikn',
accounts=[
dlz.DLzAccount(
name='development',
account_id='123456789012',
type=dlz.DlzAccountType.DEVELOP,
vpcs: [
dlz.Defaults.vpc_class_b3_private3_public(0, dlz.Region.EU_WEST_1), # CIDR 10.0.0./19
dlz.Defaults.vpc_class_b3_private3_public(1, dlz.Region.US_EAST_1), # CIDR 10.1.0./19
]
),
dlz.DLzAccount(
name='production',
account_id='123456789012',
type=dlz.DlzAccountType.PRODUCTION,
),
],
),
)
),
network={
"nats": [
{
"name": "development-eu-west-1-internet-access",
"location": NetworkAddress(
"development", str(Region.EU_WEST_1), "default", "public", "public-1",
),
"allow_access_from": [
NetworkAddress(
"development", str(Region.EU_WEST_1), "default", "private"
),
],
"type": {
"gateway": {
},
},
},
],
"bastion_hosts": [
{
"name": "default",
"location": NetworkAddress(
"development", str(Region.EU_WEST_1), "default", "public", "public-1",
),
"instance_type": ec2.InstanceType.of(
ec2.InstanceClass.T3, ec2.InstanceSize.MICRO
),
}
]
}
)Continue reading Getting Started on the DLZ documentation site.
Key Features
Notable features of the DLZ include:
- A CDK construct that can be used in TS and Python
- Opinionated but configurable and extendable. Each stack and nested component can be extended if needed.
- Leverages AWS Control Tower. Works with existing or new AWS Control Tower setups.
- Suitable for greenfield projects or integration with existing AWS resources.
- Deployable using any build systems like GitHub, GitLab, Jenkins, or locally.
- Generates compliance reports, reporting the Control Tower Standards, SecurityHub controls, Config Rules and Service Control Policies enabled in each account.
- Implements an internal wave- and stage-based deployment strategy for dependency management.
- Includes helper scripts and Standard Operating Procedures (SOPs) for routine tasks.
- Accompanied by comprehensive documentation.
Intended Audience
The DLZ handles the responsibilities and routine tasks typically managed by a Cloud Center of Excellence (CCoE) team. It's easy enough for Data Engineers to use on their own, but flexible enough to be customized and fine-tuned by experienced Cloud Engineers.
It's important to establish the following responsibilities to understand what is in and out of scope for the DLZ:
- In scope - Any CCoE responsibilities and tasks, such as account management, security, networking, compliance, etc.
- Out of scope - Application development, data engineering, and data science tasks.
Core Principles
The DLZ adheres to the following principles:
- Opinionated but configurable defaults to suit diverse needs.
- High levels of automation with manual SOPs where needed.
- Simplicity and ease of understanding over complexity
- Focused scope, limited to Landing Zone responsibilities.
Integrated AWS Services
- AWS Organizations: Seamless multi-account management.
- AWS Budgets: Track spending by tags with notifications via Slack, Teams, or email.
- Service Control Policies (SCPs): Manage service deny lists.
- Tag Policies: Enforce resource tagging for spend tracking and ownership.
- Control Tower Controls: Opinionated defaults for preventive, detective, and proactive standards.
- AWS Security Hub and AWS Config: Additional standards not covered by Control Tower. Notifications are delivered via Slack, Teams, or email
- Network Management:
- Non-overlapping VPCs across accounts.
- NAT Gateways or instances for outbound private access.
- VPC Peering for cross-account/region communication.
- Bastion Hosts with AWS SSM for secure private resource access.
- Simplified routing configurations.
- IAM Identity Center (SSO):
- User management with AWS Identity Store or external IDPs (e.g., Google, Active Directory).
- Manage Permission Sets
- Manage Access Groups to assign Users to Accounts with a given Permission Set
- Permission Boundaries: Prevent privilege escalation for all IAM roles and users.
- Configure Lake Formation for Data Lake management, including:
- General settings, like admins, version and setting hybrid iam mode
- Create Tags and specify their permissions for use within the same account and when sharing to other accounts.
- Optionally, set Tag Permissions on Tags. This is usually out of scope for DLZ, but can be configured.
How it works
The DLZ defines all accounts and resources in a single CDK construct that can be used in Typescript or Python CDK project.
The DLZ deploys resources into specified AWS accounts. These accounts must be created manually (SOP provided) in the
workloads OU, and their Account IDs must be provided in the DLZ configuration. Each account is designated
as either development or production to enable controlled, staged deployments, including the option for manual
approvals in production environments.
The construct processes the provided configuration to create logical deployment layers for resources. Each layer
consists of a Global Wave and optionally one or more Regional Waves. This structure ensures that all global stacks
for the accounts are deployed before the regional stacks. Resources in the global stack are region-independent, such
as IAM roles, whereas resources like VPCs can be deployed in both global and regional stacks if configured as such.
Continue reading the Introduction on the Data Landing Zone documentation site.
Sponsors
Proudly sponsored and created by DataChef
Contributing
See CONTRIBUTING.md for more info
Docs
Can be found at https://datalandingzone.com/
Roadmap
Can be found in the here