from my 202404 https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/v20240407/docs/google-cloud-onboarding.md
Creation of a GCP Organization including either a Workspace or Cloud Identity account - https://cloud.google.com/resource-manager/docs/creating-managing-organization. If projects and their billing ID association are created before a domain and organization are added - these projects can be migrated via https://cloud.google.com/resource-manager/docs/migrating-projects-billing
Note: this document is for initial Cloud Identity - Super Admin accounts. For production deployments, usually identity federation will be used. See Identity Onboarding and Federation options at https://cloud.google.com/architecture/landing-zones/decide-how-to-onboard-identities
When creating a new google cloud account the following artifacts will be required.
- One or more existing billing account IDs or access to a credit card that will be associated with a new billing account
- An existing or new email for use as the "Primary Admin" in https://admin.google.com/ac/accountsettings and as the "Super Admin" role in https://admin.google.com/ac/roles/53389702564151297
- An existing or new domain for organization DNS validation
There is a special IAM role that will need to be added to both any new super admin level users beyond the original SA root account and any service accounts used during automated deployments involving new projects that require associated billing permissions to link a billing account to projects. In addition to the "Project Billing Manager" we need to set the "Billing Account Administrator"
In the following screen capture - this is a new GCP account where the "root" account was automatically added to the billing permission side in billing : https://console.cloud.google.com/billing. Any additional admin users are added automatically to the billing side if they are added in IAM. Notice that the root account is not set in IAM but is set in billing.
After your organization has been created, we recommend that one of the following guardrails or landing zone deployments depending on the security profile of your projects and/or organization
https://github.com/canada-ca/accelerators_accelerateurs-gcp - 30 day Guardrails
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding - PBMM landing zone
Determining which set of guardrails or landing zone to use will depend on your cloud profile use case https://github.com/canada-ca/cloud-guardrails/blob/master/EN/00_Applicable-Scope.md#cloud-usage-profiles (Level 1 for private only sandboxes all the way to Level 6 for PBMM (Protected B Medium Integrity / Medium Availability - with or without SCED/SC2G - see slide 18-19 of https://wiki.gccollab.ca/images/7/75/GC_Cloud_Connection_Patterns.pdf)
We recommend either the 30 day Guardrails https://github.com/canada-ca/accelerators_accelerateurs-gcp or the full PBMM landing zone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding for profile 1 to 2 or prototyping work. We recommend the full PBMM landing zone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding for profile 2 to 6.
https://github.com/canada-ca/accelerators_accelerateurs-gcp - 30 day Guardrails
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding - PBMM landing zone
https://github.com/canada-ca/cloud-guardrails-gcp/tree/main/guardrails-validation - 30 day Guardrails validation
https://github.com/canada-ca/cloud-guardrails/tree/master/EN - Guardrails controls documentation
https://cloud.google.com/anthos-config-management/docs/tutorials/landing-zone - Config Controller Anthos based Landing Zone blueprint
https://cloud.google.com/docs/terraform/blueprints/terraform-blueprints#blueprints - Terraform based SDK blueprints off the Cloud Foundation Toolkit https://cloud.google.com/foundation-toolkit
Example Log Sinks https://github.com/terraform-google-modules/terraform-google-log-export
https://cloud.google.com/docs/security/infrastructure/design
https://cloud.google.com/architecture/security-foundations
https://cloud.google.com/vpc-service-controls/docs/secure-data-exchange
https://cloud.google.com/security/compliance/offerings#/regions=Canada
Navigate to admin.google.com in the security | authentication | 2-step verification
Make sure to set at least a 1 day grace period to allow the users to login to their account before adding MFA/2FA
Each user will see the following screen on signup
Follow the next steps if:
- You are a new customer and require a new domain
This scenario will guide you through the steps to purchase an available domain from Google Domains and create a new Workspace account.
Perform the following steps in an Incognito Window.
To create a new Google Workspace account follow the next steps: (WARNING) - the following link will require purchasing a Google Workspace account for the initial super admin user (the rest can be Free cloud Identity - starting at 100 users)
- For Workspace accounts go to https://workspace.google.com/business/signup/welcome.
- For Free Cloud Identity accounts use https://cloud.google.com/identity/docs/set-up-cloud-identity-admin via https://cloud.google.com/identity/docs/set-up-cloud-identity-admin - see details on 3b1: 3rd party email - free cloud identity
- Enter your Business Name (the full subdomain+domain - ie gcp.yourorg.com
- Select “Just you” under Number of employees. You can add more users later.
- Ensure “Canada” is selected as the Region.
- Click “Next”.
- Enter the info for the Google Workspace account administrator and your current email address.
- Click “Next”.
- On the next screen, select “No, I need one” to purchase a new domain.
- Search for an available domain name and click on the domain name entry or the “>” arrow.
- Click “Next”.
- Enter your business information and click “Next”.
- Select if you would like to receive tips, offers and announcements.
- Select if you would like your users to receive information and tips about Google Workspace.
- Create your first user, which will be granted the Workspace Super Admin role. Click “Agree and continue”.
- Review your payment plan and click “Next”.
- Enter the payment details and select if you would like to automatically renew your domain registration every year. Click “Next” to finalize the creation of your Workspace account.
- Click “Continue to admin console”.
Important: Check your email inbox and respond to the email asking you to verify your contact information. This is required by ICANN (the governing body for domain registration) to complete domain registration. After you purchase a domain, you'll receive an email to verify your email address. You must verify your email address within 15 days. Otherwise, your domain won't be registered and you can't use it for email and other services.
Workspace account validation steps:
- In the admin console, you will see the alert “Domain registration is pending” (as shown below) if you haven’t responded to the email that was sent to you after purchasing the domain. If you haven’t done so, please verify your email address by opening the email sent by Google Domains and clicking “Verify email now”.
- To check your service subscriptions, go to Billing -> Subscriptions. Verify you have: Domain Registration and Google Workspace.
- Go to Account -> Admin Roles to validate that your admin account was added to the Super Admin role.
The following steps will guide you through the onboarding of your GCP organization:
- Go to http://console.cloud.google.com
- Select “Canada” as Country, check the Terms of Service and click “Agree and continue”.
- Activate your Free Trial by clicking the “Activate” button at the top-right side of the screen.
- Enter the account and payment information required. These steps will set up your Billing Account.
- Answer the survey and click “Done”.
GCP validation steps:
- On the GCP console, go to IAM & Admin -> Identity & Organization.
- Click the button “Go to the checklist”.
- Ensure you have the permissions to perform certain admin actions in the console as shown below.
- Click "Cloud Identity & Organization" (on the left menu) and validate that this task/step has been completed.
- Go to IAM & Admin -> IAM to validate the permissions for your organization. Make sure that the name of your organization is selected. It should be displayed at the top of the screen, to the left of the Search field.
- Review the default roles at the organization level and grant your admin user the roles: Owner and Folder Admin.
- To validate the Folder Admin role, create a Test Folder under your organization.
- Enter "Manage Resources" on the Search field at the top of the screen. Select "Manage Resouces".
- Click "Create Folder".
- Enter the required information and make sure that your organization is selected under "Organization" and "Location". Click "Create"
- Refresh the page to see the new folder.
- On the same screen, validate that you can create projects under the folder created in the previous step.
- Click "Create Project".
- Enter the required information and make sure that your organization is selected under "Organization", and the folder previously created is selected under "Location". Click "Create".
By default, a Billing Account can only be linked to a certain number of projects, based on a variety of factors. A temporary workaround is to create additional billing accounts to get quota per account - or associate an existing billing account from another organization - see https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/docs/google-cloud-onboarding.md#onboarding-category-3b1-3rd-party-email-account---3rd-party-aws-route53-domain-validation---reuse-existing-billing-account
- default 5 per billing account - add 10-20 at a time https://support.google.com/code/contact/billing_quota_increase
- default 50 per organization - add 20-50 at a time https://support.google.com/code/contact/project_quota_increase
To submit a quota increase follow the next steps:
- try to ask for under 50 at a time - as 50 is the verification point where the automated approval may not kick in - try for 20-30
- Fill out the billing quota increase from the default 5 directly via https://support.google.com/code/contact/billing_quota_increase
- or
- Create (at least) 5 projects, or more, under the folder created in the GCP validation steps section.
- On the left menu, go to Billing and select “My Projects”. Notice that the last project has billing disabled.
- In the Actions column, click the "More Actions" (3 dots) icon corresponding to the project. Select “Change Billing”.
- Select a Billing account and click “Set Account”.
- The following message will appear. Select “Request Quota Increase”.
- Provide the required information and submit the quota increase request.
Instead of submitting a Quota Increase request, you can also create another Billing Account.
- Go to Billing, select “My Billing Accounts” and click “Create Account”.
- Provide the payment information required.
Ask for 20 from the default 5 for project/billing association - create 6 projects and assign billing on the 6th to get the popup. Select "paid services" and you will get approved in 3 min in most cases. For example
Fill out the form - note the paid services and less than 50 request
Usually you get 2 emails - the request and the response - lately we recieve just the response for billing/quota within 3 min. This is on an account that has been on past the 90 day credit period and with previously approved requests
By default your organization receives 50 licences for Cloud Identity Free edition. This will increase to 100 if you purchase services like Google Workspaces. If you require more than 50 or 100 licences and/or are prototype identity federation you can request more licenses by filling out the following form from your admin console.
- I could have used a support case instead (don't think HC has a support contract yet) - next time
- https://console.cloud.google.com/support/createcase/v2?project=biometric-ol
Navigate to admin | billing | subscriptions
-
click on the link in "Learn More"
-
-
https://admin.google.com/ac/billing/subscriptions/i9T-sfdXnZik1HvrbBnqZA
-
https://support.google.com/cloudidentity/answer/7295541?hl=en
-
Learn More | Increase your user cap | Fill out this form
The process will take around 1 hour - you may get asked for verification of your request.
This category is where the client uses their own email system but has the organization domain with GCP
Creation of a GCP Organization including either a Workspace or Cloud Identity account - https://cloud.google.com/resource-manager/docs/creating-managing-organization. If projects and their billing ID association are created before a domain and organization are added - these projects can be migrated via https://cloud.google.com/resource-manager/docs/migrating-projects-billing
Follow the next steps if:
- You are a new customer using a Gmail account with optional redirect records on an existing Google Domains hosted domain for your organization.
This scenario will guide you through the steps to create a required Cloud Identity account (using a Gmail account) and a subdomain for an existing Google Domains managed domain.
In this scenario the Gmail account is a formality. You can also use your own 3rd party email account.
Perform the following steps in an Incognito Window.
To create a Cloud Identity account follow these steps:
- Go to https://accounts.google.com/SignUpWithoutGmail.
- Enter the information required.
- Select “Create a new Gmail address instead”.
- Enter the account details.
- Click “Next”.
- Follow the steps to Verify your phone number.
- Confirm that your account has been created.
The following steps will guide you through the onboarding of your GCP organization:
-
Go to https://console.cloud.google.com and login with the account you created in the previous steps.
-
Check the “Terms of Service” and click “Agree and Continue”.
-
Go to IAM & Admin -> Identity & Organization.
-
Click “Go to the Checklist”. You will see a message stating that your current account is not associated with an organization on Google Cloud.
- Click “Begin the setup”.
- On the “Cloud Identity & Organization” screen, scroll down and select “I’m a new customer”.
- Click “Sign up for Cloud Identity”.
- On the Cloud Identity wizard, click “Next”.
- Enter the Business Name and select “Just you” under Number of employees.
- Click “Next”.
- Select the country where your business is located. Click "Next".
- Enter the Gmail account that you just created. Click “Next”. Note: You can also use your own email.
- Enter your domain name. *Note: Make sure you enter the name for a (new) subdomain (gcp.**). For example: gcp.gcloud.network
- Click “Next” to confirm the domain you want to use to set up the account. Notice the warning on email redirection - we will set this up in the domain owner account.
- Click "Next" to go to the next screen.
- You will come back to this screen after the following section.
In another window, follow the next steps to verify the domain:
- Go to https://domains.google and login with the account that owns the domain. In this case: gcloud.network.
- Select the domain, click "Manage" and go to “Email”. Notice there is no email forwarding record yet.
- Click "Add email alias".
- Enter the Email Forwarding information. Use a “super admin” alias - an account will be created later with this alias.
- Click “Add”.
- In your Gmail account inbox you will receive the following email to verify your email forwarding address.
- Click the “Verify my email now” button.
To verify the redirect follow these steps:
- Send an email to the new super admin account.
- Verify that the email was forwarded to the Gmail account.
Back on the Cloud Identity wizard:
- On the “What’s your name?” screen, enter the information for the account administrator. Click “Next”.
- Enter the username and password for the super admin account of your new subdomain.
- Select if you would like to receive tips, offers and announcements..
- Select if you would like your users to receive information and tips about Google Workspace.
- Go through the reCAPTCHA challenge and click “Agree and Create Account”.
- Click “Go to Setup”.
- Sign in using the new super admin account in your subdomain.
- Follow the steps to Verify your identity.
- Click “Accept”.
- Click “Next”.
- Click “Verify” to verify the new subdomain.
- Click “Or switch verification method”.
- Select “Create a TXT record (Recommended)”.
- Click “Next”.
- On the next screen, follow the instructions to add your verification code:
- Go to https://domains.google and login with the account that owns the domain (gcloud.network).
- Select the domain you want to verify, click “Manage” and select “DNS”.
- In the “Custom records” section, enter the host name, set Type to TXT, set TTL to 3600 (or 1 hour) and paste the TXT verification code copied previously.
- Click “Save".
- Back on the “Verify your domain” screen, click “Verify my domain”. This will take a few minutes.
- Run a dig on the subdomain.
- The Cloud Identity wizard will update when the domain has been verified.
- Click “Set up GCP Cloud Console now”. Make sure you are logged in with your new Cloud Identity super admin account.
- Check the “Terms of Service” and click “Agree and Continue”.
- Go to IAM & Admin -> IAM. Notice that the GCP organization will be automatically created.
- Click on "Select a project" at the top of the screen.
- Select the new organization in the “Select from” dropdown box.
- The new organization should be visible in the "All" tab.
Onboarding Category 3b1: 3rd party email account - 3rd party (AWS Route53) domain validation - reuse existing billing account
There are several ways to add a shared billing account - email push/pull - but the 3rd - just adding the identity user in the 2nd organization as a Billing Administrator in the organization owning the billing id works ok.
This method also reproduces the state we see where the shared billing id shows up only under “no organization : id=0” but is automatically added to new projects in the target org ok.
So we have a way to simulate the billing provisioning using 2 separate organizations.
- 20220802
- follow https://cloud.google.com/identity/docs/set-up-cloud-identity-admin and select Cloud Identity Free https://workspace.google.com/signup/gcpidentity/welcome#0
- In this case we wish to use a pre-existing billing account
- 3 personas required
-
- Cloud Identity user (a super admin (usually the first user of the target account) - the user that will onboard the organization via https://workspace.google.com/signup/gcpidentity/welcome#0
-
- Billing Account Administrator (source account) - the user who will add the account name (identity email) of the super admin of the target account above
-
- Owner of the domain zone (to be able to apply the domain verification TXT record during organization onboarding by the target super admin above)
- The Super Admin of the target account must have access to the Domain zone (even if it is sending a mail to the IT/Domain-zone owner) - to be able to set the organization subdomain TXT record for domain validation
- Billing account admin of the owning billing account must set the target Identity account as the Billing Account Administrator (full landing zone rights) - or Billing User, Billing Viewer (for single projects)
- create/use new 3rd party email account matching at least the TLD - in this case an AWS Workmail account under eventstream.io
- create new Cloud Identity account - [email protected] with domain gcp.eventstream.io - specifically via https://workspace.google.com/signup/gcpidentity/welcome#0
- validate domain via TXT record on AWS route 53
- login to the cloud console at console.cloud.google.com
- request to move billing accounts in IAM or add the target user as a billing admin in the source/owner organization
todo: caption the screencaps below
- Note we are creating a TXT record with the key set to the organization name (not related to an A or CNAME record against later workload FQDN or IP addresses)
- Note we are not creating a new domain as detailed in the process https://gc-cloud-services.canada.ca/s/dns-ground-to-public-article?language=en_US - only adding a TXT record to validate ownership
- 30-120 seconds for DNS propagation then the dialog should continue.
- Note: the billing adminstrator role must be added to the target cloud identity account by the owner of the billing id - this is done in the "manage billing" section of billing - separate from adding the role in IAM.
We see the no organization issue - no problem we can still use the account for the main organization
This category is common for organizations new to GCP or multicloud where both the email system and DNS hosting zone are 3rd party
See the similar section Onboarding Catagory 3 - GCP hosted domains
Usually copy/paste or email
- using the original super admin/owner create another cloud identity account with an email on the organization domain - with optional email forward to their work email. Give rights such as "Owner" or "Folder Admin" to this 2nd+ user, when they login to console.cloud.google.com they will already have proper access to the organization (no domain validation required)
goto the admin page at admin.google.com
Add the new user - using an existing super admin user
send login instructions - with temp password
Start witn an incognito chrome window
launch accounts.google.com
Login to new user
new account splash
auto change password
view new account
select profile picture on top right - add (to get a new chrome profile for the user)
login again
accept profile
Navigate to the cloud at console.cloud.google.com
Accept the license
View that you are already on the existing organization (no DNS verify required)
Attempt to create a project - switch to the org
select the organization - normal without a higher role we will set with the super admin user
verify you don't have rights yet to the organization
check the onboarding checklist to verify
Yes, you don't have the rights yet
Switch tabs to the other super admin user - goto IAM to verify roles
Add the new user to the role of "Owner" for now - normally use "Folder creator" and "Organization Administrator" for example
Verify the user 2 role change
back at user 2 navigate to IAM | cloud identity | verify your new rights
Notice you now have rights to the organization - good to go
This category is a variant of category 3 where there is a gmail account with option redirect where the organization zone records are on a 3rd party DNS system
This category is common for individual consumers where they do not have a gmail account or any domain. This option will not have an organization top node in IAM
Onboarding Category 9: Gmail Email or Google Account on 3rd party email or proxy - no Domain - no Organization
This category is common for individual consumers where they gmail account but no domain, or a 3rd party domain where they have an email or proxy but do not wish to setup or validate a domain. This option will not have an organization top node in IAM.
With a Google Account only - you can still collaborate across separate Google Accounts in GCP by sharing resources via IAM.
All the organization policies are still available at the project level.
Later you may migrate these NO ORGANIZATION project into the organization created at a later date via https://cloud.google.com/resource-manager/docs/project-migration
Procedure:
create google account on existing email address - can also be a domain email proxy https://support.google.com/accounts/answer/27441?hl=en https://accounts.google.com/signup
create new profile - login navigate to gcp - notice no org - accept credits https://console.cloud.google.com/getting-started
Create an email alias or use an email that does not already have a google account
verify via 2FA to your phone
Optional accept credits and setup credit card - or associate an existing billing account later with your project
Google Cloud Identity accounts are ideal for cloud account organizations where the user identities are maintained outside of Google cloud in for example AWS Workmail or Azure Active Directory.
Create or gain access to the domain you wish to associate or federate users from. For example packet.global.
You will need access to the domain zone to add TXT records for domain validation under a subdomain like gcp.packet.global
Open Chrome Window with no Google Account
Onboarding to Google Cloud using a cloud identity account and a 3rd party managed domain - AWS Route53
.. continuing from "open chrome window" above
Launch SignUpWithoutGmail - select gmail
https://accounts.google.com/SignUpWithoutGmail
Select gmail, register and launch a new browser - add new account - login
Create your Google Account (gmail)
launch google cloud
https://console.cloud.google.com/
do not select an org yet - as the domain under GCP registration does not have an email yet and is not registered with workspace.
You will not be able to run the organization checklist account as a gmail user - https://console.cloud.google.com/cloud-setup/organization
Add Cloud Identity free in
https://cloud.google.com/identity/docs/set-up-cloud-identity-admin
follow
https://workspace.google.com/signup/gcpidentity/welcome#0
add your gmail address and GCP domain
Add email capability https://support.google.com/cloudidentity/answer/7667994
Select the email left tab on https://domains.google.com/registrar/eventstream.dev/email?hl=en-US
Select email forwarding to to your gmail account
Launch gmail to verify email - don't worry it will launch domains in your current gmail account - verify that the verify worked in your other account that holds the domain registration
image
Check email forwarding on the DNS tab
image
wait for DNS record propagation 30 sec and recheck the cloud identity wizard warning on no email MX records
image
continue wizard regardless of warning - use your new email forward address
https://workspace.google.com/signup/gcpidentity/tos
goto setup after creation
image
Launch admin
Since I have used this phone a couple times - get past the unusual activity dialog
Identity account OK
select getting started
https://admin.google.com/u/1/ac/signup/setup/v2/gettingstarted
Verify domain - sign in option will not work on this browser - as I have it registered on another account - in this case select "Switch Verification Method" and select the 2nd TXT option.
add the TXT record
Click Verify back on the admin page
The org in this case will automatically create when you click the link below (no subdomain as the TXT record is the first on the domain. If there is already a root domain TXT record - you will need to use a subdomain like gcp.domain.com
org is setup as the TXT record is against the root domain on the separate GCP account
https://accounts.google.com/SignUpWithoutGmail
Fill in the form with an existing email address outside of Google
Launch from step 2 of the IAM | Cloud Identity & Organization | checklist https://console.cloud.google.com/cloud-setup/organization
to https://workspace.google.com/signup/gcpidentity/welcome
Following is an example manually created landing zone infrastructure - however continue to use this landing zone for production environments.
1 - as original root super admin user
1a - create root organization on cloud login - done above
admin-root permissions
Folder Admin
Organization Administrator
Owner
1b - create user acc-1 in admin.google.com
navigate to http://admin.google.com
Dont worry about saving the password - we wil reset it
1c - add acc-1 user to super admins
1d - add acc-1 user IAM roles
navigate to http://console.cloud.google.com - search on IAM and switch the project dropdown to the organization
Billing Account Administrator
Folder Admin
Organization Administrator
Organization Policy Administrator
Click add - and start typing acc in the principle or past the entire email of the acc user
Add Billing Account Administrator, Folder Admin, Organization Administrator, Organization Policy Administrator and hold off on Owner and Folder Admin.
1e - add extra billing accounts - or do in step 2d
2 - as acc-1 user
Create a new Chrome profile and login as acc-1@domain
2a - create business folder at root
Open http://console.cloud.google.com
Switch to the organization in IAM
Goto resouce manager off IAM
2b - create project business-unit off business folder
You will need to search on the new folder
Note: projects must be globally unique - append the first chars of your domain name to differentiate - here nuage-cloud = nc
2c - create users bus-1 and dev-1 in admin
Login to http://admin.google.com
Reset passwords
2d - add bus-1 and dev-1 IAM roles
Login as acc-1 user in http://cloud.google.com
bus-1 has (BigQuery Admin, Billing Account Administrator, Compute Admin, Compute Network Admin, Folder Admin, Logging Admin, Monitoring Admin, Networks Admin, Project Billing Manager, PubSub Admin Security Admin, Storage Admin, Tag Administrator)
Billing Account Administrator
Folder Admin
Logging Admin
Monitoring Admin
Networks Admin
Project Billing Manager
Security Admin
Storage Admin
Tag Administrator
dev-n has to start (BigQuery Admin, CloudSQL Admin, Compute Admin, Compute Network Admin, Networks Admin, Network Management Admin, PubSub Admin, Storage Admin)
BigQuery Admin
Cloud SQL Admin
Compute Admin
Compute Network Admin
Logging Admin
Monitoring Admin
Network Management Admin
Pub/Sub Admin
Source Repository Administrator
Source Repository Writer
Storage Admin
Viewer
For multiple accounts - use a group email in Admin and target the group account in IAM or use a custom IAM role composed of individual roles
Create group in admin for developers - add bus-1 as the owner add dev-1/2 as members
Add the above roles for dev-1/2 to developers@domain in IAM
3 - as bus-1 user
Create Chrome Profile and login to http://cloud.google.com
Switch to the org
Add Project Billing Manager role if missed above to IAM permissions
3a - create folders sandbox and project off business folder
3b - create project deployment-1 off project folder
3c - create project pipeline-1 off project folder
3d - create project sandbox-1 and 2 off sandbox folder
3e - associate billing account 2 and 3 with sandbox 1 and 2
Create 3rd billing account before associating sandbox-2 if different billing accounts needed. Note: if you change the contact email away from the default a decision may take 48h
4 - as dev-1 user
Even though i reset the password of this user - the new Chrome profile forced me to change it this time. The dev user also shows up with the credit dialog
Select the org in IAM - verify restricted permissions
Select the project you have access to in IAM - verify IAM is ok
Add project viewer if not already applied
4a - create specific infrastructure in sandbox-1 project
Verify no access to projects outside your scope
Add source.repos.create permissions in bus-1
Add source repository admin - to be able to create new CSR repos
verify Permissions on the sandbox project for the developers group in bus-1
Verify access to projects inside your scope
Create the CSR
Verify billing in bus-1 is set to a different account for the sandbox-1 project
4b - use specific intrastructure in deployment-1 and pipeline-1 project
5 - as dev-2 user
5a - create a cloud run deployment from an existing container in deployment -1
During testing for the following section https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/docs/google-cloud-onboarding.md#onboarding-category-3b1-3rd-party-email-account---3rd-party-aws-route53-domain-validation---reuse-existing-billing-account
On occasion you will get the following DENY https://workspace.google.com/signup/gcpidentity/deny on creating a cloud identity user during organization onboarding using the procedure in https://cloud.google.com/identity/docs/set-up-cloud-identity-admin if you repeatedly use the same cloud identity creation process for the same domain
using for example a couple attempts on
The primary workaround is to contact your FSR or CE and/or support to get your domain on an allowlist. The SLO for this is usually under 24 hours.
The secondary workaround is to use a separate TLD domain and subdomain for now. The full workaround is TBD (time based, Google Support unflag...TBD). The fact that the dialog states that your computer may be compromised is very likely not the issue as I have registered another cloud identity account right after on the same machine/browser. The issue looks to be domain related on domains new to google cloud that have had several attempts at creating a cloud identity account on the same domain.
I retested this particular domain for timing and using a different email, subdomain and also look for a workaround and/or support fix - will try different computer/5G location for the flagged domain.
After the domain allowlist entry - Cloud Identity onboarding proceeded OK.
https://workspace.google.com/signup/gcpidentity/done
- 20220809: TBD - document any procedure to create an organization without access to the actual domain - where TXT record submission to the zone is not possible. I would expectd that this is variant use case and could be used to add subdomain to a domain the client does not own - hence private zone access only in this case. However there are cases where the user has not yet gained access to the domain zone in their org and wishes to create/validate the domain for a new organization before actuall domain validation can be done.
Onboarding 12: New Cloud Identity users are flagged as User Suspended by default in admin security alert center - ignore - this is a red-herring
- 20220902: We will get to the root cause and determine the criteria for default suspension when creating a new org or importing identity users - for now you can ignore or reset the suspension (note: we need to know when the suspension is real)
- For example this org was onboarded from scratch and the super admin idenity user was already flagged as "User Suspended" - with no effects.
- see GoogleCloudPlatform/pubsec-declarative-toolkit#292
- see BAA requirement for shared billing - https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/issues/255
- Verify billing quota first
- Type 1: shared billing account where account owner in other org adds the super admin account in this org as a Billing Account Administrator and/or Billing Account User where normal IAM inheritance into Billing is not done due to security separation. In this case the target service account must have a copy of it's Billing Account User role also set on the billing page under the org "NONE SELECTED".
- Type 2: direct billing credit card on this account (all tests above so far are this case) State of billing id associations for type 2 are the following (this one is for the guardrails install canada-ca/accelerators_accelerateurs-gcp#47) - notice that the terraform service account is in the list as well as the user super admin account.
TL;DR; Shared billing accounts do not get shared IAM roles - they need to be set separately
We need a workaround (see GoogleCloudPlatform/pbmm-on-gcp-onboarding#177 ) for the fact that if the billing account is of type "shared" - owned by a source organization where it comes in under the target organization as "Non Selected, ID=0" then any service account created will not get inherited links from IAM set in Billing - these like Billing Account User - need to be set manually. The workaround is currently manual - set the billing role directy in Billing on the shared account. See IAM Role inheritance into Billing Roles in https://cloud.google.com/billing/docs/how-to/billing-access
Example
michael@cloudshell:~$ gcloud config set project gcp-zone-landing-stg
Updated property [core/project].
michael@cloudshell:~ (gcp-zone-landing-stg)$ export PROJECT_ID=$(gcloud config list --format 'value(core.project)')
michael@cloudshell:~ (gcp-zone-landing-stg)$ export ORG_ID=$(gcloud projects get-ancestors $PROJECT_ID --format='get(id)' | tail -1)
michael@cloudshell:~ (gcp-zone-landing-stg)$ export SA_PREFIX=tfsa-example
michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud iam service-accounts create "${SA_PREFIX}" --display-name "Terraform example service account" --project=${PROJECT_ID}
Created service account [tfsa-example].
michael@cloudshell:~ (gcp-zone-landing-stg)$ export SA_EMAIL=`gcloud iam service-accounts list --project="${PROJECT_ID}" --filter=tfsa --format="value(email)"`
michael@cloudshell:~ (gcp-zone-landing-stg)$ echo $SA_EMAIL
[email protected]
check existing roles
michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud organizations get-iam-policy $ORG_ID --filter="bindings.members:$SA_EMAIL" --flatten="bindings[].members" --format="table(bindings.role)"
Set the billing role
gcloud organizations add-iam-policy-binding ${ORG_ID} --member=serviceAccount:${SA_EMAIL} --role=roles/billing.user
check again
michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud organizations add-iam-policy-binding ${ORG_ID} --member=serviceAccount:${SA_EMAIL} --role=roles/billing.user
Updated IAM policy for organization [925207728429].
...
michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud organizations get-iam-policy $ORG_ID --filter="bindings.members:$SA_EMAIL" --flatten="bindings[].members" --format="table(bindings.role)"
ROLE: roles/billing.user
It may take a couple min to show in IAM
Checking billing on the shared account
expected on billing accounts belonging to this org - via IAM inheritance in billing
not expected on billing accounts shared from other orgs
Workaround - set manually
ref GoogleCloudPlatform/pbmm-on-gcp-onboarding#177
For most cases we do not modify the original owning organization on a particular billing account. When we want to distributed this BID (billing ID) to other organizations in the form of project/billing associations - we do this via the Cloud Identity roles "Billing Account Administrator" - which can also assign "Billing Account User" roles to service accounts or just "Billing Account User". The identity user or SA in the receiving org can then switch projects to this shared billling account.
However there is a way to move a billing account (not recommended for cloud brokerage shared billing client organizations). When the organization admin role is assigned in the BID owning org to the organization admins / billing account admins in the target or sub-orgs (remember all organizations are flat - but they can appear as sub-orgs via subdomains).
In the "Billing Account Management" view - select "Change Organization" and select one of the sub-orgs like below
Moving the billing account between organizations is usually reserved for full organization migration.
- Note: billing accounts and shared billing accounts cannot be deleted - they can be removed from an org by removing the "Billing Account Administrator" role for other organization super admins (not the current owned billing project for this org)
- see https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/docs/google-cloud-onboarding.md#shared-billing-accounts
- Summary: delete the project to ensure resources are shutdown
- Disabling billing will remove billable
There are 4 scenarios to start 1 - project(s) deletion - https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects 2 - project migration to another organization - https://cloud.google.com/resource-manager/docs/project-migration and https://cloud.google.com/resource-manager/docs/moving-projects-folders 3 - project backup/restore (at the GCP service level using GCE snapshot for example, or if the services were raised using gcloud/terraform/kcc deployment scripts) 4 - disassociate billing on the project
- two organizations (gcp.zone - org with the project to be deleted) and (landing.gcp.zone - org owning the shared billing account)
- Shared billing account from landing.gcp.zone to gcp.zone org via Billing Account Administrator role
- https://console.cloud.google.com/billing?organizationId=962342543445&supportedpurview=project
-
- Project to be deleted in gcp.zone associated with landing.gcp.zone billing
- https://console.cloud.google.com/billing/projects?organizationId=925207728429&supportedpurview=project
-
- Current bill was running a total of $85 for clouddeploy-gz over this month
-
- notice that project clouddeploy-gz is assocated with the billing account 0127C1-...-EA25D6 from another org - the goal is to shutdown completely the project and disassociate billing so no more charges occur
- We navigate to the project in the owning organization and delete the project - starting a 30day cycle where it can be reactivated
- follow https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects
- Quote "If the project has a billing account associated with it, that association is broken, and isn't reinstated if the project delete operation is canceled. After 30 days, the project is fully deleted."
- via https://console.cloud.google.com/cloud-resource-manager?organizationId=925207728429&supportedpurview=project
-
- hit delete on selected project
-
- wait -w e will check out removing/disassociating biling first below
- Back up from removing billing - we readded billing back and will delete the project
-
-
- Project "clouddeploy-gz" is now shut down and scheduled to be deleted after Nov 26, 2022.
-
- nothing off the billing page anymore
I will add gcloud scripting shortly
- Alternatively disable billing on the project without deleting it - in billing (project will be both deleted and visible in IAM this way)
-
- or select billing on the dropdown in resource manager
-
-
-
-
- disable billing "When you disable billing, any billable activity of your services will stop, your billable resources may be deleted, and your application may stop functioning."
-
- Billing is disabled
-
- Project is still there but services are shutdown (almost the same as project deletion)
-
- Only default/uncharged services are still up in IAM - Asset Inventory
-
- You can still shell into your project
-
- But you cannot create billable resources - where we used to run a double GKE cluster - we cannot create GCE VMs or use GKE for example
-
-
- either way - the project is deleted - whether you delete it or remove billing - both remove billing and place the project in "resources pending deletion"
-
- see https://support.google.com/googleapi/answer/6251787?hl=en#zippy=%2Crestore-a-project
- Note that even though I have project creator and deleter roles on my SA - I cannot undelete a previously deleted or billing associated project without adding resourcemanager.projects.undelete or owner
- There may be a residual bill for essentially empty accounts - of 0.46 - investigating how to remove this charge
-
- in my case I have 2 projects that are billing under 0.01 per day with a cumulation of .23 and .21 = .44 - with 13% tax up to .49 - not all services are regional so if the buckets are in NA they will be taxed. Anyway I will determine which resources are having resdidual - likely cloud storage bucket costs.
-
Note: Deleted projects are recoverable up to 30 days from "pending deletion" state - after that they are fully deleted from backup. billing-disabled/unset projects stay in IAM and have active free resources (such as an IAM service account) | Manage Resources - and are visible in /billing/projects indefinitely (IE: I have non-empty billing disabled projects from 2014 in one of my orgs) Therefore I recommend projects be in "Deleted" not "Disable Billing" state - unless you want to reenable them after 30 days.
For example an older billing disabled project
The project below has been in billing disabled state since 2014
I do however have a left over Google App Engine application (from an Eclipse in-IDE deployment) left over from 2019 that is created but not deployed - hence no billing account required yet.
You can see that the service account created for GAE at 9AM on 2 May 2019
This section details procedures around deleting one or more organizations. The use case around org deletion is usually in the context of semi-automated sub-org creation/deletion as teams iterate in and out of project level GCP deployments in a multi-org multi-tenant landing zone structure.
Note:
- billing data is on a 24h refresh cycle
- Deleting an organization resource https://cloud.google.com/resource-manager/docs/creating-managing-organization
- Delete your organization's Google Account https://support.google.com/a/answer/9468554?hl=en
There are two organizations involved in this use case - the org under deletion gcp...network and the org holding the billing id gcp...network.
We will need to delete the super admin reference to the org under deletion that may still may be associated as a billing account administrator on the organization holding the billing account this is step 2 in the "creating-managing-organization" section
- historical billing - org still up - org BAA still associated (normal scenario)
- historical billing - org still up - org BAA unassociated more than 24h ago
- historical billing - org deleted more than 24h ago - org BAA still associated
- historical billing - org deleted more than 24h ago - org BAA unassociated more than 24h ago
Scenario: we will delete the organization gcp...network after at least 24h of billing association with the owning org approach...zone
- OA_ORG_D: Organization Administrator account (and Workspace super admin) - ad-s@gc*.g*.n*
- ORG_D: Organization to delete = gcp.*.network
- BID_1: Organization owning BID 1 = app*.gcp.z*
- BID_2: Organization owning BID 2 = gcp.z*
- The org admin OA_ORG_D for ORG_D is currently associated as a BAA under BID_1 and BID_2
- delete 3 remaining projects including 2 stopped GCE VMs on ORG_D
- Note: billing should be disabled first - if not goto billing | account managment - disable billing on deleted projects
- dev-* with BID_1 app*
- gcloud-* with BID_2 gcp*
- (optionally disassociate OA_ORD_D from BID_1 and BID_2 - if you have BAA access to these other organizations
- verify no projects in billing | my projects
- verify single billing account directly under the org in billing
- verify 3 billing accounts under the org "none selected"
- delete all subscriptions except (cloud identity)
- follow https://support.google.com/a/answer/9468554?hl=en
- navigate to Account | Account settings | Account Management | Delete Account
Delete account
Check account
- attempting to login = "This account was recently deleted and may be recoverable. Click Next to attempt to restore this account."
- check shared billing at BID_1 and notice the BAA association now appends the following to the deleted account "uid=5034....247802"
- We will wait 24h to verify whether billing history stays up and also verify whether the BAA assocation deletion has an effect on billing history (deletes historical records) - by removing the BAA only from BAA_2
- To verify whether the GCP super admin gcloud/cloud-identity account has been deleted - check role associations in https://console.cloud.google.com in any other organization linked to the account and look for a ?uid=.... appended to the email
- Before
- on g*.z* removing billing association on a deleted project
- removing the BAA role on the deleted account
- in 24h check historical billing still shows for this deleted account
delete principal for "Billing Account Administrator" role on the owning organization of one of the billing accounts.
Principal "[email protected]?uid=503...47802" successfully removed from all policies on resource "billingAccounts/01...05.."
We still have historical billing data - will recheck after 24h
In the shared billing account owning organization you will continue to see historical billing metrics around any projects, folders and the organization itself after deletion.
- see historical organization data under billing | Reports | 90 days
- specifically here gcp...services is historical but gcp...network is a currently billing project
- in the report view you will notice that we still show gcp...services org project traffic-os up to Dec 2 (it is Dec 23rd in this capture) - even though the org
- in the report view note that a 3rd org landing...zone that had billing traffic in Nov under the fortigate project shows billing data even though the organization SA was removed from the billing account administrator role on the owning org approach...zone more than 24h ago.
the landing...zone SA is not in the BAA list below
Muliti organization single pane is possible (flat model) pending details....
