Skip to content

Instantly share code, notes, and snippets.

@houey
Last active September 14, 2024 09:13
Show Gist options
  • Save houey/fa1129edb2214f1d278010578ea29c18 to your computer and use it in GitHub Desktop.
Save houey/fa1129edb2214f1d278010578ea29c18 to your computer and use it in GitHub Desktop.

Preface:

Its unfortunately extremely common for customers and enterprises operating in AWS to have chosen a workload/storage bearing account (more than likely, the main production account) as the Organization Management Account (formerly known Organization "Master" account, before AWS adopted better naming).
Many customers and companies operating in AWS made this decision in 2018 or so and its unforunately not something that can be easily changes as of today (2024). Many customers have requests to AWS to make a friendly path for rehoming the Org Management account but last I heard it is still not prioritized. Thus, we as customers are left to go through the nerve-wracking, if not dangerous process of migrating to a new AWS Organization in order to align with modern best practices and reduce common privilege escalation and account to account lateral movement concerns (made worse if you happen to have enabled things like Cloudformation Stacksets, Control Tower, or other powerful services in the same account where your workloads run (ec2,ecs,eks,lambda) or common storage resides (s3,RDS,DynamoDB))

We went through this process at the tail end of 2022 for a well know digital-native enterprise operating in AWS. Here are my notes. Keep in mind, there are are NOT one size fits all. We were fortunate that we had not at the time adopted Identity Center, RAM/VPC Sharing, or policies that used PrincipalOrgId/ResourceOrgId with any major volume. Please plan accordingly!

Also, if you have access to AWS Support.. file tickets and work with them.. get your name/company behind the enhancement request to allow rehoming of the Organization Management account.. AWS needs to make this easier for all of us!


Org Migration notes

Here is the high-level plan we adopted:

Create new, dedicated Org Management Account and Cloudtrail OrgTrail.

Create OU structure (this is a religious debate, stay as flat as you can but I like to leave Level2 branch in case of aquisition.. ie Bank decides to acquite Cat Grooming service)

Create baseline SCPs. (These are the "easy ones" defang root, no deleting cloudtrail, dont use c3 instances, restrict IAM User creation, and a bunch more)

Set Up new Orgtrail to deliver to current delegated Log Archive bucket..dont want to lose cloudtrail history

Decommission Control Tower Landing Zone. (this is what we did.. and we never turned it back on)

Coordinate with AWS to ensure no customization specific to company is lost in the migration. They made this blog post way after we were done.. has some tips: https://aws.amazon.com/blogs/mt/aws-organizations-moving-an-organization-member-account-to-another-organization-part-1/

Coordinate with CostOps/Finance/Capacity Engineering team(s) to ensure billing continuity is maintained in the migration

YOU WILL LOSE NATIVE BILLING HISTORY WHEN THE OLD ORG DIES. Prepare for that if it is important.

Update OrganizationAccountAccessRole (or equivalent) in all member accounts to be assumeable from the new Org Management account

Send invites to join the new Org to all of the legacy member accounts. (you will have to do the Legacy Org Management acct last, after you delete the legacy organization)

Use the OrganizationAccountAccessRole or equivalent to perform migration ( ie, leave Org then Accept new Org Invites) where possible to avoid Root Usage.

Update any principalOrgId or resourceOrgId references in existing SCP, IAM, resource based or VPCE policies to accept the old and new org id.

If you have ever shared AMIs by Organization. Keep this in mind as you move the account that houses the original.

If you use RAM for shared resources/shared VPCs. Take the time to move accounts together, if possible.

Move smaller, non-business-critical accounts to new Org

Move most remaining accounts to new Org

Move legacy OrgManagement account to new Org


Some questions worth considering:

How do we guarantee that our complete pricing and discount regime survives the migration without interruption? For a brief period that account will be "standalone":

Are there any potential timing issues to consider when executing the migration process for an account (detach account from original org, attach account to new org)?

Will we need to supply temporary payment information in order to allow accounts to be detached from the original org? YES (Work with AWS account team and "concierge team" to update billing across all accounts ahead of migration, if that is available to you. Otherwise you may need to go put a creditcard on file in every account. (yuck)

Are there any custom, Org-level API rate limits, resource quotas, etc that could potentially be lost, disrupted or reset in the migration? (When building the new ORG mirror any Org level Service limit/quotas ahead of time

If you use Identity Center in the legacy org... talk to AWS. We did not use it at the time. If you are planning a move to Identity Center.. consider an org migration before!

for Control Tower users: Control Tower complicates the org migration process. Recommend decommissioning the landing zone

this goes hand in hand with any current AWS Config usage that is aggregated or Security Hub Aggregation.

The best time to do the Org Migration is always yesterday.

If you are starting down a regime of VPCe, RAM, Service Control Policies.. its nearly impossible to have proper security posture if you are managing the Org from an account that also serves workloads or stores data that is accessed in any way.

I opted to mirror my Cloudformation StackSets in the new Organization.

I opted to make a staging OU in the old AWS Organization such that when AWS Account were getting ready to move they first went to the staging OU which would strip all the current stacksets from the account.

When they migrated and landed in the OU for the new Org the stacksets would again Apply and build the resources.

CAUTION If you depend on cross account roles generated by stacksets in various policies, this can be a breaking change as the IAM Identifier will change and you will need to update those policies with the ARN (again).

Plan and test, work with AWS

@nwachonky
Copy link

Hello houey,
Great article and this came at the right time for me because am also looking into some migration.
you mentioned you opted to mirror Cloudformation Stackset into the new ORG, was it because your stakset were not self-managing ?
Only Control Tower CF are SERVICE_MANAGED for me and since that will be decommissioned am not worried
i will appreciate some feedback

@houey
Copy link
Author

houey commented May 10, 2024

Hello houey, Great article and this came at the right time for me because am also looking into some migration. you mentioned you opted to mirror Cloudformation Stackset into the new ORG, was it because your stakset were not self-managing ? Only Control Tower CF are SERVICE_MANAGED for me and since that will be decommissioned am not worried i will appreciate some feedback

I recall a few factors.

  1. there were a large handful of Stacksets that were there for projects/things that were abandoned. So removing those old Stacksets was good hygiene.
  2. I considered just migrating the accounts.. this would disconnect the resources from cloudformation stack. There is some less than simple way to import those orphaned resources into a new stack. I couldn’t quite figure it out and was fortunate that the few move and create fresh in the new org method was doable.
    I used a “to be migrated” OU. Would remove the stacks via shell script. Then migrate the account. In the new org the stackset were applied to the sub OUs and not to root. (The helped with how we subset accounts)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment