Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does the Read Permisisons stack support mutiple data collection accounts? #1137

Closed
work-bw opened this issue Mar 3, 2025 · 5 comments
Closed

Comments

@work-bw
Copy link
Contributor

work-bw commented Mar 3, 2025

I want to send my data collection data to multiple data collection accounts.

In the management account, can I use a have multiple data collection account ids separated by commas in the deployment parameters of the Read Permissions stack? Or do I need to have multiple installations of the Read Permission stack template as it only supports one data collection account id? Thanks.

Brian

@iakov-aws
Copy link
Collaborator

Hello,
Please can you elaborate on the usecase? Why 2 data collection accounts?

@iakov-aws
Copy link
Collaborator

you can have 2 difereent data colleciton accounts with different prefixes collecting data from your one or many organizations

OR you can think about s3 data sync with external accounts, depending on what is the desired outcome and restricitons you have

@work-bw
Copy link
Contributor Author

work-bw commented Mar 3, 2025

Hello, Please can you elaborate on the usecase? Why 2 data collection accounts?

I have two CID environments.

CID environment 1= Lab setup where I can safely test upgrades and customization. 1 org (OrgA) to 1 data collection account in OrgA.
CID environment 2= Production environment where I have significant customization. 9 orgs (OrgA, OrgB, OrgC, etc) to 1 data collection account in OrgB.

Today, I have CUR replication to both CID environments. I want to do the same configuration with data collection.

Because I already have the ReadPermissionStack for CID environment 1 configured for OrgA, I'm trying to figure out how to add CID environment 2. Can I modify the existing ReadPermissionStack that goes to CID environment 1 to also send the same data to CID environment 2? Or do I need to have another ReadPermissionStack2 that points to CID environment 2?

===========

My ultimate goal is to get multiple orgs sending data collection data to CID environment 2. My understanding is that I need the ReadPermissionStack in the management accounts and I need 1 CidDataCollectionStack in the data collection account to receive all the data from the management accounts.

Then all the data will be in S3:cid-data-XXXXXXXXX/organizations/organization-data/payer_id=account 1 and payer_id=account 2, etc, where the state machines will read it.

I assume the state machines parse the data by payer_id so that the dashboards can filter by payer_id.

Brian

@iakov-aws
Copy link
Collaborator

You can have 2 Permissions stacks with different prefixes. But it will mean that you will need to maintain and update 2 also there will be 2 times API calls.

Other option for you is to add an S3 replication from your PROD account to your LAB account. This way data will be collected just once and then replicated.

We do not have it out of the box using s3 replication but you can do that. You can use s3 replication or some glue job that will be totally outside of CID stacks.

@iakov-aws
Copy link
Collaborator

I will close this for now. If any follow up question please contact us to set a meeting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants