Skip to content

Commit

Permalink
feat: enable google site analytics
Browse files Browse the repository at this point in the history
  • Loading branch information
fethiozdol committed Jul 2, 2024
1 parent 56d9ae7 commit da9e736
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 46 deletions.
10 changes: 5 additions & 5 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ keywords: aws, xecuta, rpa, rpa-farm, terraform # add your own keywords or leave
lang: en # the language of your site (for example: en, fr, cn, ru, etc.)
icon: ⚛️ # the emoji used as the favicon (alternatively, provide image name in /assets/img/)

url: https://fethiozdol.github.io # the base hostname & protocol for your site
url: https://thisismypattern.com # the base hostname & protocol for your site
baseurl: # the subpath of your site, e.g. /blog/. Leave blank for root
last_updated: false # set to true if you want to display last updated in the footer
impressum_path: # set to path to include impressum link in the footer, use the same path as permalink in a page, helps to conform with EU GDPR
Expand Down Expand Up @@ -118,12 +118,12 @@ contact_note: >

# For Google Analytics, see https://support.google.com/analytics/answer/10447272?hl=en&ref_topic=14088998&sjid=5129943941510317771-SA#zippy=%2Cgoogle-sites
# and follow the instructions for Google Sites. You will need to create a Google Analytics property and copy the Google tag ID.
google_analytics: # your Google Analytics measurement ID (format: G-XXXXXXXXXX)
google_analytics: G-BFVSGSW9BJ # your Google Analytics measurement ID (format: G-XXXXXXXXXX)
cronitor_analytics: # cronitor RUM analytics site ID (format: XXXXXXXXX)
pirsch_analytics: # your Pirsch analytics site ID (length 32 characters)

# For Google Search Console, see https://support.google.com/webmasters/answer/9008080?hl=en#meta_tag_verification&zippy=%2Chtml-tag
google_site_verification: # your google-site-verification ID (Google Search Console)
google_site_verification: 5Lhp1lN5mNB84LS6y_8GsdoHcwSb29xxGr1LlOvbF5w # your google-site-verification ID (Google Search Console)
bing_site_verification: # out your bing-site-verification ID (Bing Webmaster)

# -----------------------------------------------------------------------------
Expand Down Expand Up @@ -417,10 +417,10 @@ lazy_loading_images: true # enables lazy loading of images (recommended)
# Optional Features
# -----------------------------------------------------------------------------

enable_google_analytics: false # enables google analytics
enable_google_analytics: true # enables google analytics
enable_cronitor_analytics: false # enables cronitor RUM analytics
enable_pirsch_analytics: false # enables Pirsch analytics (https://pirsch.io/)
enable_google_verification: false # enables google site verification
enable_google_verification: true # enables google site verification
enable_bing_verification: false # enables bing site verification
enable_masonry: true # enables automatic project cards arrangement
enable_math: true # enables math typesetting (uses MathJax)
Expand Down
79 changes: 38 additions & 41 deletions _posts/2024-07-01-ht-startup-w-devops-p01.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,29 +21,24 @@ related_posts: false

# Foreword

Perhaps it's the banking industry in which I started my IT career, but I've been obsessed with high speed and high quality software delivery ever since I took up platform engineering activities.
Perhaps it's the banking industry in which I started my IT career, but I've been obsessed with high speed and high quality in software delivery ever since I took up platform engineering activities.

Like at any startup, there are few people writing code, and it may be OK for some to leave software delivery disorganized. Not for us.

We build multiple SaaS for businesses at [Xecuta](https://www.xecuta.co.uk) and it was obvious from day 0 that we had to set our solid architecture principles.

It may be our early days, but our architecture principles have already given us the confidence in managing AWS resources effectively.
We build multiple SaaS for businesses at [Xecuta](https://www.xecuta.co.uk). We have only few people writing code, like at any startup, but it was not OK for us to leave software delivery disorganized. It may be our early days, but our architecture principles have already given us the confidence in managing AWS resources effectively.

In the first part of this series, I'll talk about our architecture principles and our design thinking which led us to organize our AWS environments and choose tools and services.

## Xecuta's Architecture Principles

Here's our 10 architecture principles that we agreed before we wrote our first line of code.
Here's our 10 architecture principles that we agreed before we wrote first line of code.

1. Build cloud-native and serverless architectures, because we want maximum agility and minimum OpEx from day 0 with a near-linear trajectory for OpEx:Revenue ratio at short-to-medium long term.
2. Build services powered by Amazon Web Services (AWS), because they provide the best public cloud services around cloud-native and serverless architectures.
3. Organize our AWS environment using AWS Organizations and adopt AWS recommended [Basic Organization with infrastructure services](https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/basic-organization.html#basic-organization-with-infrastructure-services) pattern
1. Build cloud-native and serverless architectures, because we want maximum agility and minimum OpEx from day 0 with a near-linear trajectory for OpEx:Revenue ratio at short-to-medium long term
2. Build services powered by Amazon Web Services (AWS), because they provide the best public cloud services around cloud-native and serverless architectures
3. Organize our AWS environment using AWS Organizations
- We adopted AWS recommended [Basic Organization with infrastructure services](https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/basic-organization.html#basic-organization-with-infrastructure-services) pattern
4. Use infrastructure-as-code for not only anything-on-cloud, but also to provision and manage

- AWS accounts
- Git repositories
- CI/CD pipelines

- AWS accounts
- Git repositories
- CI/CD pipelines
5. Security from day 0
- Use SSO to access anything
- Use free or low-cost security services where possible
Expand All @@ -56,11 +51,11 @@ Here's our 10 architecture principles that we agreed before we wrote our first l
- Multi git-repo strategy with isolated and independent CI/CD pipelines
- Re-build and re-deploy untouched code every 3 months
10. Build once deploy many with immutable build artefacts
- Use conventional commits at all times
- Always use conventional commits
- Use automation for semantic versioning and generating CHANGELOG
- Use package managers like npm or pip to manage code dependencies

> Re: #3: You may want to look at [Basic organization with CI/CD as a separate function](https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/basic-organization.html#basic-organization-with-cicd-as-a-separate-function) as well, but we decided to make Infrastructure OU enclose Deployments OU for simplicity.
> Re: principle #3: You may want to look at [Basic organization with CI/CD as a separate function](https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/basic-organization.html#basic-organization-with-cicd-as-a-separate-function) as well, but we decided to make Infrastructure OU enclose Deployments OU for simplicity.
## How it's going so far

Expand All @@ -80,20 +75,20 @@ At the time of writing, Xecuta has:
- Policy-as-code for infrastructure-as-code
- 7,000+ lines of all code

> We are just 1 full-time and 1 part-time developers \
> AND AWS bill for **non-workload OUs** last month was only **$19.90**
> We are just 1 full-time and 1 part-time developers. \
> AWS bill for **non-workload OUs** last month was only **$19.90**.
---

## Plan from Day 0

First, let's talk about how we organize our AWS accounts and resources.

Assume we have a single AWS account, which becomes our AWS Organizations management account and we configure our IAM Identity Center to enable SSO.
Assume we have a single AWS account, which becomes our AWS Organizations management account, and we configure our IAM Identity Center to enable SSO.

With respect to our principles #3 and #4, we want all organization OUs and member accounts to be managed by infrastructure-as-code.

For this purpose, we have 2 single infrastructure-as-code Git repositories:
We have 2 single infrastructure-as-code Git repositories for this purpose:

- AWS Accounts Management
- Git repositories Management
Expand Down Expand Up @@ -133,9 +128,9 @@ For the time being, let's assume we have a Shared Services account, it has the b
Our design thinking on how we should organize our AWS resources led us to several questions, such as:

- If a KMS key is used to encrypt and decrypt messages across all SQS queues in an AWS account, where should the key be maintained and who should maintain it
- There may be some AWS services like VPCs that our developers can be hands-off and we may want to restrict their maintenance to a smaller set of people, such as DevOps engineers only
- We must avoid cyclic dependencies and how can we control this in an ecosystem with many microservices
- We may want to keep the number of dependencies between microservices at a minimum so they continue to be easily disposable
- There may be some AWS services like VPCs that our developers can be hands-off, or we may want to restrict their maintenance to a smaller set of people, such as DevOps engineers only
- How can we control dependencies in an ecosystem with many microservices and avoid cyclic dependencies
- How can we keep the number of dependencies between microservices at a minimum, so they continue to be easily disposable

**Workload Infra Base** is our answer to this problem.

Expand Down Expand Up @@ -168,7 +163,7 @@ Assume we have a CRM system, in which Customer Service exposes an API for our we

In another scenario, we may want to have a centralised Email microservice in our Core Services, and we'd like it to use SES identity domain, which is environment-agnostic therefore managed outside our workloads, in our Shared Services.

In short, resources can be both intra-workload and inter-workload dependencies.
In summary, AWS resources can be intra-workload or inter-workload dependencies, or both.

```mermaid
flowchart TD
Expand Down Expand Up @@ -199,27 +194,29 @@ classDef cmp color:#fff, fill:#b509ac, stroke:#303;

---

## Inter-Workload and Intra-Workload Resource Dependencies
## Inter-Workload and Intra-Workload Dependencies in Infrastructure-as-Code

Terraform has quickly become the de-facto in infrastructure-as-code but it does not fill the gap between _infrastructure_ and _application code_, and that's expected, because it's not designed for software development.

We can use AWS CDK as the singular approach to implement the infra base and components. Since AWS CDK creates CloudFormation stacks, it is possible to import & export resources between CloudFormation stacks, exporting from infra base's stacks and importing into components' stacks.
We can use AWS CDK as the singular approach to implement the infra base and components. Since AWS CDK creates CloudFormation stacks, it is possible to share resources between CloudFormation stacks by exporting values from infra base's stacks and importing them into components' stacks.

Whilst this approach has its many pros, there are some important drawbacks of using AWS CDK from our perspective:

1. Import & Export functionality in CloudFormation creates physical dependencies between CloudFormation stacks, and this may cause human interventions during major refactoring activities. Imagine logging into prod environment and deleting CloudFormation stacks manually. That's not going to work for us.
2. Cloud Development Kits like AWS CDK have brought the power of programming languages into infrastructure-as-code, enabling full stack engineering and rapid software development. Enterprises can produce reusable and modular CDK modules as `npm`, `pip`, or `go` packages, but there aren't many reusable CDK modules available in the wider community, whereas there are many production-ready modules available in Terraform today.
3. This is my personal opinion and I'm sure many of you will disagree with this, but Id don't buy into using programming languages for infrastructure-as-code. Although I've been using AWS CDK for my other projects lately, I'm still an advocate of using more limited and strict languages like YAML for infrastructure-as-code. I believe "The best code is no code at all" is true for infrastructure-as-code. Even the simplest programming language brings some complexity overhead; you need more knowledge to write good code in a programming language and you also need to perform maintenance for runtime environment upgrades. With HCL or YAML, there is not much of a major room for any runtime upgrades or weird runtime errors or any unintended behaviours.
1. Import & Export functionality in CloudFormation creates physical dependencies between CloudFormation stacks, similar to the referential integrity concept in RDBMS. CloudFormation refuses to delete the exported resources until no other CloudFormation stack imports them. This approach will cause human interventions eventually during major refactoring activities. Imagine logging into prod environment and deleting CloudFormation stacks manually. That's not going to work for us.
2. Cloud Development Kits like AWS CDK have brought the power of programming languages into infrastructure-as-code, enabling full stack engineering and rapid software development. Enterprises can produce reusable and modular CDK modules as `npm`, `pip`, or `go` packages, but there aren't many reusable CDK modules available in the wider community. On the other hand, there are many production-ready modules available in Terraform today.
3. This is my personal opinion and I'm sure many of you will disagree with this, but I don't buy into using programming languages for infrastructure-as-code. Although I've been using AWS CDK for my other projects lately, I'm still an advocate of using more limited and strict languages like YAML for infrastructure-as-code. I think "The best code is no code at all" is a good saying for infrastructure-as-code. No matter how simple a programming language is, it always brings some complexity overhead. With HCL or YAML, there is not much of a major room for any runtime upgrades or weird runtime errors or any unintended behaviours:
- You need knowledge to write good code in a programming language and you need to replenish your knowledge occassionally because programming languages evolve continuously
- Programming language releases have an expiry date, and that means you will need to perform maintenance or even refactor code if new versions have any breaking changes.

Third is a personal opinion. We could have lived with the first drawback, but the second one is the deal breaker. We're quite limited on resources (people and time); we can't maintain 1000s of lines of infrastructure-as-code to reinvent the wheel.
Third is an opinion and we could have lived with the first drawback, but the second one is the deal breaker. We're quite limited on resources (people and time), and we can't maintain 1000s of lines of infrastructure-as-code just to reinvent the wheel.

An alternative could be "CDK for Terraform", but we are not comfortable with living on the cutting edge; as CDKTF "may still have breaking changes before its 1.0 release".
An alternative could be also "CDK for Terraform", but we are not comfortable with living on the cutting edge as CDKTF "may still have breaking changes before its 1.0 release".

We could also look into [Pulumi](https://www.pulumi.com/docs/concepts/vs/terraform/). Its features appear to be very promising, but Terraform's de-facto position within the wider community, was the tiebreaker for us. Limitless reusable modules, and not to mention our existing Terraform knowledge.
We could also investigate [Pulumi](https://www.pulumi.com/docs/concepts/vs/terraform/). Its features appear to be very promising, but Terraform's de-facto position within the wider community, is still the tiebreaker for us. With Terraform we have a limitless number of reusable modules available, and not to mention that our existing Terraform knowledge helped our delivery rate in early days.

---

Obviously, Terraform would not be good enough on its own for software development and testing on our local workstations, so we started experimenting with a hybrid-solution to get the best out of two different worlds:
Obviously, Terraform would not be good enough on its own for software development and testing locally, so we started experimenting with a hybrid-solution to get the best out of two different worlds:

- We use Terraform to build and manage our workloads' infra base
- We use AWS SAM to **build and package but _not to deploy_** our serverless applications
Expand All @@ -234,16 +231,16 @@ Obviously, Terraform would not be good enough on its own for software developmen
Swipe left to see how our pipelines work in our hybrid solution.
</div>

Thanks to trunk based development, strict conventional commits and automated semantic versioning, our builds produce immutable builds that we build once and deploy many. CD pipelines understand the target environment from the artefact version and perform Terraform plan & apply, which actually does CloudFormation stack create/update in the background.
Thanks to trunk-based development, strict conventional commits and automated semantic versioning, our builds produce immutable builds that we build once and deploy many. CD pipelines understand the target environment from the artefact version and perform Terraform plan & apply, which does CloudFormation stack create/update in the background.

Finally, if the Workload Infra Base gets very big, we can split into 2 or more repositories and orchestrate their deployments using [Terragrunt](https://terragrunt.gruntwork.io/).
Finally, if the Workload Infra Base gets very big, we can segregate into 2 or more repositories and orchestrate their delivery using [Terragrunt](https://terragrunt.gruntwork.io/).

We were quite happy with our experiment and decided to follow this pattern in building SaaS at Xecuta. I understand our approach may seem too complex or unnecessary for some, but again, best out of two worlds!
We were quite happy with our experiment and decided to follow this pattern in building SaaS at Xecuta. I understand our approach may seem too complex or unnecessary for some, but again, best out of two worlds:

- Thanks to Terraform, we are able to reuse terraform modules produced by the community and this keep LoC of our infrastructure-as-code low
- Thanks to AWS SAM, we can develop and test locally
- And we can manage inter-workload and intra-workload dependencies in a consistent and reliable manner
- Thanks to Terraform, we can reuse terraform modules produced by the community and/or build our own private modules, resulting in low and modular LoC for our infrastructure-as-code
- Thanks to AWS SAM, we can develop serverless applications and test them locally
- We can still manage inter-workload and intra-workload dependencies in a consistent and reliable way

I hope you enjoyed my first blog post about Workload Infra Base pattern with hybrid implementation using AWs SAM and Terraform.

Part 2 of this series will continue with which SCM and Orchestration tool(s) we opted to use and how we built our Shared Services initially. Until then, ciao!
Part 2 of this series will continue with which SCM and Orchestration tool(s) we opted to use and how we built our Shared Services initially.

0 comments on commit da9e736

Please sign in to comment.