AWS Landing Zone Configuration Nightmare: Tales from the Trenches

As an AWS Community Builder, I dove headfirst into setting up a secure AWS environment. My mission? Deploy a Trusted Enclaves landing zone with Entra ID integration, leveraging my $500 AWS credit. What followed was a hilarious (in hindsight) odyssey of errors, timeouts, and head-scratching moments. Buckle up for a cautionary tale about AWS Landing Zone deployments gone wrong, and learn how to avoid my mistakes!

The Grand Plan: So Simple, So Wrong

My initial approach was deceptively straightforward:

Use my AWS account and Community Builders credit.
Deploy AWS Landing Zone Accelerator with Control Tower.
Apply the Trusted Enclaves configuration.
Integrate with Entra ID for seamless access.
Document the whole process.

Easy peasy, right? Wrong.

The Quota Quest Begins: A 12-Hour Wait for More Accounts

The first hurdle? Service quotas. Deploying an AWS Landing Zone requires an increased account quota. Twelve-hour waits are never fun, but this was just the beginning of my troubles.

SCP Chaos: When Security Policies Attack

After finally clearing the quota hurdle, I deployed stage one and applied the Trusted Enclaves configuration. That's when the SCPs hit the fan. The configuration attempted to apply a sixth Service Control Policy (SCP) to an OU, but AWS has a default limit of five.

Thinking I was clever, I removed the direct all-access SCP, only to trigger a cascade of errors. The Landing Zone deployment stalled, and new accounts lost trust relationships with the audit and master billing accounts. Hours were spent drowning in CloudWatch logs, attempting to manually fix cross-account trust relationships with no success - this is a common problem when configuring an AWS Landing Zone.

The Nuclear Option: Starting Over… Almost

Admitting defeat, I decided to nuke the entire setup and start fresh. I needed to retain my master billing account (for that sweet credit), so the plan was to:

Close all child accounts.
Remove all resources (using bash scripts for mass termination).
Restore the master account to its pristine state.
Try again.

However, the AWS environment had other plans, the Landing Zone Accelerator stubbornly detected my suspended accounts as active and refused to re-deploy.

Account Purgatory: Suspended in Limbo

Stuck between a rock and a hard place, I needed to unsuspend the accounts to remove them correctly. Cue another support ticket and another agonizing 24-hour wait. Once unsuspended, the problems intensified, with accounts refusing to leave the organization, so the Trusted Enclaves configuration refused to finish running, this is a common problem when configuring an AWS Landing Zone in a multi-account environment. Despite providing billing information, they simply refused to leave the organization.

Hard-Earned Lessons for Your AWS Journey

So, what did I learn from this painful, yet strangely enlightening, experience?

Service Quotas are Key: Request quota increases well in advance. Don't wait until the last minute to request quota increases.
SCPs are Deceptive: Understand SCP limits before modifying them. Test your Service Control Policies in a dev environment!
Document Everything: Track every change, error, and support response. You'll thank yourself later.
Account Lifecycle Matters: AWS accounts have complex states that can affect operations in unexpected ways. Don't delete and recreate accounts in the same hour.
Time Estimate? Double It: What I thought would be a weekend project turned into a multi-week saga. Allow leeway in your project timelines.

Have you deployed an AWS Landing Zone with enhanced security controls? What challenges did you face? Tell me your tales of AWS woe! Let's find solace in shared experience. I would love to hear how other AWS engineers navigated the complex maze of an AWS Landing Zone Deployment.