Skip to content

Commit

Permalink
Updated the documentation and added the software capability.
Browse files Browse the repository at this point in the history
  • Loading branch information
danielscholl committed Jan 17, 2024
1 parent cb0d91f commit b9372fb
Show file tree
Hide file tree
Showing 32 changed files with 780 additions and 25 deletions.
73 changes: 50 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,91 +2,118 @@

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

This project is intended to provide a simple way of providing a mechanism to develop for OSDU using the Azure Cloud.

## Project Principals
[Open Subsurface Data Universe](https://osduforum.org) (OSDU) is a standard data platform that brings together a diverse array of subsurface and well data. It enables the energy industry to access and analyze data across various sources efficiently. This project aims to provide a streamlined approach for developing and working directly with [OSDU](https://community.opengroup.org/osdu/platform) using the [Azure Cloud Platform](https://azure.microsoft.com/).

The guiding principal we have for this project is to focus on providing a solution to allow for an easy way to develop for OSDU using the Azure cloud. It is not intended to support any kind of a production scenario and no support for this solution is provided. It is built with the following 2 pillars of the [Azure Well-Architected-Framework](https://learn.microsoft.com/en-us/azure/well-architected/what-is-well-architected-framework) kept in mind.

1. Cost Optimization -- A cost optimized solution with cost in mind but accepting the tradeoff of security.
2. Security -- The intent is to provide a feature enabled solution to increase levels of security as best as possible.
## Project Principles

Additionally, the solution is desired to be easy to use and to support that is built with support for [Github Codespaces](https://github.com/features/codespaces) along with the [Azure Developer CLI](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/).
The guiding principle of this project is to offer an accessible solution for facilitating direct engagement with the OSDU codebase on Azure in a minimal fashio. This solution is not intended for production use and does not come with official support. Our approach aligns with two key pillars from the [Azure Well-Architected Framework](https://learn.microsoft.com/en-us/azure/well-architected/what-is-well-architected-framework):

1. Cost Optimization -- We aim to create a cost-effective solution, balancing cost with security considerations.
2. Security -- Our goal is to enhance security levels within the constraints of a development-focused solution.

To support ease of use, the project integrates closely with [Github Codespaces](https://github.com/features/codespaces) and the [Azure Developer CLI](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/), facilitating seamless development and innovation on the OSDU platform.


## Setup

__Features__
### Registering Azure Features

This solution makes use of features in Azure that are in Public Preview and might require some features to be registered for use.
This solution utilizes Azure features that are currently in Public Preview. Certain features need to be registered before use.

Register the _AzureServiceMeshPreview_ feature flag by using the az feature register command:
**Step 1: Register the AzureServiceMeshPreview feature**
Use the `az feature register` command to register the _AzureServiceMeshPreview_ feature flag:

```bash
az feature register --namespace "Microsoft.ContainerService" --name "AzureServiceMeshPreview"
```

It takes a few minutes for the feature to register. Verify the registration status by using the az feature show command:
It may take a few minutes for the feature to register.


**Step 2: Verify the Registration Status**

Confirm the registration status using the az feature show command:

```bash
az feature show --namespace "Microsoft.ContainerService" --name "AzureServiceMeshPreview"
```

When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider by using the az provider register command:
Look for a status that indicates Registered.

**Step 3: Refresh the Resource Provider**

Once registered, refresh the Microsoft.ContainerService resource provider:

```bash
az provider register --namespace Microsoft.ContainerService
```

This solution uses Resource Group Scoped Deployments which is an Alpha Feature for the Azure Developer CLI.
### Enabling Alpha Features for Azure Developer CLI

**Resource Group Scoped Deployments**

This solution uses Resource Group Scoped Deployments, an Alpha Feature in the Azure Developer CLI.

Enable this feature using the following command:

```bash
azd config set alpha.resourceGroupDeployments on # Enable Alpha Feature
```

__Login__
Note: Alpha features are experimental and might be subject to changes. Use them with this consideration.

Log into the Azure CLI from a command line and set the subscription.
If running with windows ensure that Azure Powershell is connected as well.

### Authentication

The Azure Developer CLI requires authentication. Log in using the following command:

```bash
azd auth login
```

__Environment Variables__

An environment must be created using the following environment variables.
### Environment Variables

**Environment Variables**
Set up the environment using the following variables. You can find these values in your Azure portal or by using appropriate Azure CLI commands.

An environment must be created using the following environment variables.

| Variable | Purpose |
| :-------------------- | :-------------------------------------- |
| AZURE_SUBSCRIPTION_ID | The Azure Subscription _(GUID)_ |
| AZURE_LOCATION | The Azure Region |
| AZURE_CLIENT_ID | Azure AD Application Client Id _(GUID)_ |

Initialize the environment and set the Azure Client ID:

```bash
azd init -e dev

APP_NAME= # <-- <your_ad_application_name>
APP_NAME= # <-- <your_ad_application_name>

azd env set AZURE_CLIENT_ID $(az ad app list --display-name $APP_NAME --query "[].appId" -otsv)
```

### Workspace

The developer workspace is brought online using the azure developer cli

## Workspace

The solution template is provisioned using the azure developer cli.

| Action | Command |
| :----- | :------------------------- |
| Start | `azd up` |
| Stop | `azd down --purge --force` |


### Infrastructure

The following diagram repesents the infrastructure that is deployed by this solution.

![[0]][0]
_Architecture Diagram_


## Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a
Expand Down
1 change: 0 additions & 1 deletion software/README.md

This file was deleted.

5 changes: 5 additions & 0 deletions software/applications/dev-sample/namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: dev-sample
44 changes: 44 additions & 0 deletions software/applications/dev-sample/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: env-tool
namespace: dev-sample
spec:
chart:
spec:
chart: ./charts/env-debug
sourceRef:
kind: GitRepository
name: osdu-bicep
namespace: flux-system
interval: 5m0s
install:
remediation:
retries: 3
targetNamespace: dev-sample
values:
replicaCount: 1
nameOverride: ""
fullnameOverride: env-debug
service:
type: ClusterIP
port: 80
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 3
targetCPUUtilizationPercentage: 80
azure:
enabled: false
env:
- name: MESSAGE
value: Hello World!
# - name: AZURE_TENANT_ID
# secret:
# name: active-directory
# key: tenantid
# - name: WORKSPACE_ID
# secret:
# name: central-logging
# key: workspace-id
117 changes: 117 additions & 0 deletions software/applications/elastic-search/elastic-search.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# This sets up an Elasticsearch cluster with 3 nodes.
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: elasticsearch-ha
namespace: elastic-search
spec:
version: 8.5.3
nodeSets:
- name: master
count: 3
config:
node:
roles: ["master"]
store.allow_mmap: false
podTemplate:
spec:
tolerations:
- effect: NoSchedule
key: app
value: "elasticsearch"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: agentpool
operator: In
values:
- espoolz1
- espoolz2
- espoolz3
- matchExpressions:
- key: topology.kubernetes.io/zone
operator: In
values:
- southcentralus-1 # THIS DEPENDS ON LOCATION
- southcentralus-2 # THIS DEPENDS ON LOCATION
- southcentralus-3 # THIS DEPENDS ON LOCATION
env:
- name: ES_JAVA_OPTS
value: "-Xms2g -Xmx2g"

- name: coordinating
count: 3
config:
node:
roles: []
store.allow_mmap: false
podTemplate:
spec:
tolerations:
- effect: NoSchedule
key: app
value: "elasticsearch"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: agentpool
operator: In
values:
- espoolz1
- espoolz2
- espoolz3
- matchExpressions:
- key: topology.kubernetes.io/zone
operator: In
values:
- eastus2-1 # THIS DEPENDS ON LOCATION
- eastus2-2 # THIS DEPENDS ON LOCATION
- eastus2-3 # THIS DEPENDS ON LOCATION
env:
- name: ES_JAVA_OPTS
value: "-Xms2g -Xmx2g"

- name: data
count: 6
config:
node:
roles: ["data"]
store.allow_mmap: false
volumeClaimTemplates:
- metadata:
name: elasticsearch-data
spec:
accessModes:
- ReadWriteOnce
storageClassName: es-storageclass
podTemplate:
spec:
tolerations:
- effect: NoSchedule
key: app
value: "elasticsearch"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: agentpool
operator: In
values:
- espoolz1
- espoolz2
- espoolz3
- matchExpressions:
- key: topology.kubernetes.io/zone
operator: In
values:
- eastus2-1 # THIS DEPENDS ON LOCATION
- eastus2-2 # THIS DEPENDS ON LOCATION
- eastus2-3 # THIS DEPENDS ON LOCATION
env:
- name: ES_JAVA_OPTS
value: "-Xms2g -Xmx2g"
21 changes: 21 additions & 0 deletions software/applications/elastic-search/kibana.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
name: kibana
namespace: elastic-search
spec:
version: 8.5.3
http:
tls:
selfSignedCertificate:
disabled: true
elasticsearchRef:
name: "elasticsearch"
count: 3
podTemplate:
spec:
tolerations:
- key: "sku"
operator: "Equal"
value: "elasticsearch"
effect: "NoSchedule"
4 changes: 4 additions & 0 deletions software/applications/elastic-search/namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: elastic-search
20 changes: 20 additions & 0 deletions software/applications/podinfo/ingress.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: podinfo
namespace: podinfo
spec:
hosts:
- "*"
gateways:
- aks-istio-ingress/istio-external-gateway
- aks-istio-ingress/istio-internal-gateway
http:
- match:
- uri:
prefix: "/"
route:
- destination:
host: "podinfo.podinfo.svc.cluster.local"
port:
number: 9898
7 changes: 7 additions & 0 deletions software/applications/podinfo/namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v1
kind: Namespace
metadata:
name: podinfo
labels:
toolkit.fluxcd.io/tenant: dev-team
istio.io/rev: asm-1-18
27 changes: 27 additions & 0 deletions software/applications/podinfo/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: podinfo
namespace: podinfo
spec:
releaseName: podinfo
chart:
spec:
chart: podinfo
sourceRef:
kind: HelmRepository
name: podinfo
interval: 50m
install:
remediation:
retries: 3
# Default values
# https://github.com/stefanprodan/podinfo/blob/master/charts/podinfo/values.yaml
values:
redis:
enabled: true
repository: public.ecr.aws/docker/library/redis
tag: 7.0.6
ingress:
enabled: true
className: nginx
Loading

0 comments on commit b9372fb

Please sign in to comment.