From 971dff18b633619afd3bf531afa566c4d6f2a70c Mon Sep 17 00:00:00 2001 From: svrnm Date: Wed, 19 Jun 2024 10:18:48 +0200 Subject: [PATCH 1/3] satisfy markdown linter, remove aspecto references Signed-off-by: svrnm --- CODE_OF_CONDUCT.md | 4 +- README.md | 19 +++++- apiserver/README.md | 22 +++++-- docs/DeployEnvoyFilterIstio.md | 36 +++++++---- docs/Go&TinyGo.md | 9 +-- docs/InstallingIstio.md | 17 +++-- docs/Istio&WASMBasics.md | 4 +- docs/OPA&Istio.md | 95 ++++++++++++++++------------ docs/OpenTelemetryBasics.md | 72 ++++++++++----------- docs/UserAccountCRUD.md | 18 ++++-- docs/Using-lineage_propagation.py.md | 78 ++++++++++++----------- docs/VSCode Server.md | 27 ++++---- docs/WASM-Label-Lineage.md | 78 +++++++++++++---------- docs/WASMPluginFromScratch.md | 23 ++++--- docs/WritingOPAPolicies.md | 14 ++-- mysqldb/REAME.md | 18 ++++-- wasm-lineage-headers/README.md | 14 ++-- 17 files changed, 322 insertions(+), 226 deletions(-) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 8af3c79..8caad38 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -71,9 +71,9 @@ members of the project's leadership. ## Attribution This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, -available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html +available at [homepage]: https://www.contributor-covenant.org For answers to common questions about this code of conduct, see -https://www.contributor-covenant.org/faq + diff --git a/README.md b/README.md index a1d1b88..2af9736 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # Path Warden + This project looks to improve the security in cloud systems. Specifically, it demonstrates: + 1) A system for tracing the Lineage of data moving through services 2) A strategy for storing the Lineage information for a piece of data 3) An enforcement point for pieces of data based on their Lineage @@ -8,6 +10,7 @@ This project looks to improve the security in cloud systems. Specifically, it de This design looks at solving these issues in dynamically constructed edge systems. ## Overview + So far, there are 4 key components to this system: 1. Lineage Propagation: This refers to the propagation of labels related to a piece of data moving through a set of services. The entirety of this set of labels is the `Data Lineage`. The original source of the data is the `Data Provenance`. The goal of lineage propagation is to save the original Data Provenance and concatenate onto it each processing step the data undergoes to generate a Data Lineage that can be evaluated at each proceeding step and ultimately stored for future reference. @@ -18,14 +21,17 @@ So far, there are 4 key components to this system: 4. Policy Evaluation: At the enforcement points, not-yet-evaluated labels are sent off to a separate service to be evaluated for their pass/fail status. Otherwise, previously cached values are simply retrieved. -#### Key Technologies & Concepts +### Key Technologies & Concepts + - Label Propagation built using OpenTelemetry - Enforcement done in Istio Sidecars using Go Wasm Plugin - Policies written in Rego & evaluated using OPA - It is necessary that developers instrument their applications with our lineage propagation (tracing) libraries. However, these libraries desire to be incredibly lightweight and easy to use. ### Current State + #### Summary of System + 1. Label Propagation is achieved using Open Telemetry's Baggage Concept. We store a Label Set in JSON format at the baggage labeled `lineage_label_set`. 2. Label Storage is achieved in myqsql by creating a separate table whose primary key is equivalent to the primary key of the table one is labeling. The current example shows this being done using a small library of functions in python. This allows enabling/disabling the labeling of data in existing systems without updating/destroying existing tables. 3. We enforce data label policies in Istio's Service Mesh sidecars using a Go-Wasm Plugin. Reference Istio & the Go Wasm SDK for more information on those. @@ -38,14 +44,17 @@ So far, there are 4 key components to this system: - wasm-lineage-headers contains all files relevant to the development of the plugin written for the Istio sidecar which parses, validates & caches LabelSets. ##### (Provided as Reference) + - OTel Basic contains a Series of Services used to Initially Develop & Test OpenTelemetry. Generally, the functionality created here is less mature than that in account-CRUD. These files are provided for general reference. - OPA contains files relevant to testing & developing the OPA implementations. Ultimately, the plug-and-play solution of OPA for Istio was not used however these files are provided as reference. #### Implementation Summary + As mentioned before, account-CRUD contains the demo of this system. See the README in that directory. ### Pre-reqs + - minikube installed on system - Istio installed on Minikube cluster - gsutils installed @@ -53,25 +62,31 @@ As mentioned before, account-CRUD contains the demo of this system. See the READ - tinygo installed #### Recommended Additional Software + - VSCode Server on instance for remote IDE access #### Getting Up and Running with Cluster on EC2 instance + 1. Create Tunnel from terminal: `ssh -L 8080:localhost:8080 ` 2. Launch VSCode server: `code-server --auth none` 3. Open new Terminal on local machine & ssh: `ssh ` 4. Create Tunnel from EC2 instance to minikube gateway `minikube tunnel` 5. Launch New Terminal & ssh which serves as your working cli -#### Accounts Required For: +#### Accounts Required For + - GoogleCloud: Remotely storing & Deploying the Wasm Plugin using a GC bucket. Archived files show how it can be deployed with a local file (see envoyFilter.yaml). Deploying EnvoyFilter with GoogleCloud means saving it to a ConfigMap and injecting ConfigMap into app deployment. See docs for more information. - Docker: if you want to push images. Not necessary. - Use Jaeger for OpenTelemetry trace visualization ##### Early Steps + In addition to connecting & establishing the minikube tunnel, early on you will need to do the following things: + 1. Should you ever need to edit/redeploy the wasm-lineage-headers plugins (which is very likely), you'll need to make a google cloud account, and a bucket to contain the wasm file. Reference the git linked at the top of `WASM-Label-Lineage.md`. ### Future Work + - More Languages supported for Label Propagation - More Databases supported for Label Storage - Integrate OPAL for simpler Policy distribution diff --git a/apiserver/README.md b/apiserver/README.md index 23d9819..81a5449 100644 --- a/apiserver/README.md +++ b/apiserver/README.md @@ -1,22 +1,30 @@ # Flask API + ## 1. Description + Base application for API Server ## 2. Deployment + In order to ensure the good functioning of this component service the following steps should be undertaken in the specified order: - - Deploy MySQL database (see _db/README.md_) - - kubectl apply -f _../OTel_Basic/otel-agent-config.yml_ - - docker build . -t _/flask-api-server_ - - kubectl apply -f _./flaskapi-deployment.yml_ - - kubectl apply -f _./laskapi-configmap.yml_ - - kubectl apply -f _./flaskapi-service.yml_ + +- Deploy MySQL database (see _db/README.md_) +- `kubectl apply -f ../OTel_Basic/otel-agent-config.yml` +- `docker build . -t /flask-api-server` +- `kubectl apply -f ./flaskapi-deployment.yml` +- `kubectl apply -f ./laskapi-configmap.yml` +- `kubectl apply -f ./flaskapi-service.yml` ## 3. Automated Build + Please check the flaskapi-deployment.yml, line 21, and fill in with your account username by which you're going to build the docker image for the flask api server. -The default _devacct_ value is _$USER_. +The default _devacct_ value is _$USER_. The image can be automatically build by running: + - ./build.sh _devacct_ ## 4. Automated Start + This service can be brought up automatically by running; the script doesn't include OpenTelemetry Agent Configmap: + - ./start.sh diff --git a/docs/DeployEnvoyFilterIstio.md b/docs/DeployEnvoyFilterIstio.md index 76a9f16..5f6a999 100644 --- a/docs/DeployEnvoyFilterIstio.md +++ b/docs/DeployEnvoyFilterIstio.md @@ -1,55 +1,68 @@ +# Deploying an EnvoyFilter resource into istio + The following instructions are for deploying an EnvoyFilter resource into istio. This is generally useful as Istio's custom WasmPlugin resources is missing a number of features/configuration options for Wasm plugins. Notably: + - The default WasmPlugin resource doesn't support TCP filters - The default WasmPlugin resource doesn't allow for configuring vm_id We use an EnvoyFilter resource to deploy the WASM plugin in the account-CRUD demo. -Furthermore, this tutorial focuses on deploying using a local wasm file and a ConfigMap. While not recommended due to the fact this approach requires modifying the manifest of the pod you are deploying into, we provide it here as reference. +Furthermore, this tutorial focuses on deploying using a local wasm file and a ConfigMap. While not recommended due to the fact this approach requires modifying the manifest of the pod you are deploying into, we provide it here as reference + --- Learnings based off: -https://sirishagopigiri-692.medium.com/deploying-envoy-filter-on-istio-ce2d2573b981 + See Tutorial Above for Full explanation. Below are some useful/notable steps and commands ## Pre-Reqs + - Istio Installed - Namespaces Labelled: `kubectl label namespace default istio-injection=enabled` - Create & Write Wasm Plugin (See WASMPluginFromScratch.md) ## Deploying + ### Create Config Map in Default Namespace: only done once + `kubectl create configmap NAME-filter --from-file=NAME-filter.wasm=main.wasm` - + #### Modify Application yml Deployment: only done once unless file is edited -* In the Deployment section of the App, under `spec.template.metadata` add these annotations: + +- In the Deployment section of the App, under `spec.template.metadata` add these annotations: + - `sidecar.istio.io/userVolume: '[{"name":"NAME-filter","configMap":{"name":"NAME-filter"}}]'` - `sidecar.istio.io/userVolumeMount: '[{"mountPath":"/var/local/wasm","name":"NAME-filter"}]'` - `sidecar.istio.io/logLevel: "info"` - + #### Redeploy Service + `kubectl apply -f .yml` - + ##### Can view annotations w/ + `kubectl describe pod POD-NAME|grep -i NAME-filter` - + **Testing:** You can test Service Response now w/ wtv curl command you have. Just to ensure you have connectivity before we deploy the Plugin. **Get Logs for Plugin** `kubectl logs -f -l app=NAME-c istio-proxy` -**Deploy EnvoyFilter** +**Deploy EnvoyFilter** `kubectl apply -f filter.yml` **Verify deployment** `kubectl get envoyfilter` --- + Here I have provided a manifest file for an HTTP as well as a TCP EnvoyFilter. Both of these were tested and worked when deploying using the strategy outlined in the article. -# HTTP Filter +## HTTP Filter + ```yaml apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter @@ -86,7 +99,8 @@ spec: runtime: envoy.wasm.runtime.v8 ``` -# TCP Filter +## TCP Filter + ```yaml apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter @@ -118,4 +132,4 @@ spec: local: filename: /var/local/wasm/.wasm allow_precompiled: true -``` \ No newline at end of file +``` diff --git a/docs/Go&TinyGo.md b/docs/Go&TinyGo.md index 3c160a3..a190252 100644 --- a/docs/Go&TinyGo.md +++ b/docs/Go&TinyGo.md @@ -1,9 +1,10 @@ -**Installing Go** -https://go.dev/doc/install +**Installing Go** + + - Use `wget` to download tar file - Use sudo into tar **Installing TinyGo** -https://tinygo.org/getting-started/install/linux/ -- Remember to add Path to .bashrc! + +- Remember to add Path to .bashrc! diff --git a/docs/InstallingIstio.md b/docs/InstallingIstio.md index 46b4041..819059d 100644 --- a/docs/InstallingIstio.md +++ b/docs/InstallingIstio.md @@ -1,29 +1,34 @@ # Installing Istio -## Docs: -https://istio.io/latest/docs/setup/getting-started/#download +## Docs + + ### Summary of the Above + `curl -L https://istio.io/downloadIstio | sh -` `cd istio-1.16.1` `export PATH=$PWD/bin:$PATH` #### Adding permanently to path + `vim ~/.bashrc` Paste at bottom: `export PATH=/home//istio-1.16.1/bin:$PATH` - + ### Continuing Istio Install + `istioctl install --set profile=demo -y` + - Do Not Run this. Provided for reference: `istioctl uninstall -y --purge` - *Apply this at your discretion but it's more than likely it's what you want* kubectl label namespace default istio-injection=enabled + - Do Not Run this. Provided for reference: `kubectl label namespace default istio-injection=disabled` - + **If you already have pods installed/running, you'll likely need to restart the pods so that the Istio proxy can get injected properly** `kubectl delete --all pods --namespace=default` **Verify there are no Istio issues** -`istioctl analyze` \ No newline at end of file +`istioctl analyze` diff --git a/docs/Istio&WASMBasics.md b/docs/Istio&WASMBasics.md index 98a91e5..26b5943 100644 --- a/docs/Istio&WASMBasics.md +++ b/docs/Istio&WASMBasics.md @@ -1,4 +1,4 @@ ## Tutorial - + Simply Follow this for a basic Understanding of WASM Plugins in Istio: -https://tetrate.io/blog/istio-wasm-extensions-and-ecosystem/ \ No newline at end of file + diff --git a/docs/OPA&Istio.md b/docs/OPA&Istio.md index 7bfa80a..eb931b7 100644 --- a/docs/OPA&Istio.md +++ b/docs/OPA&Istio.md @@ -1,16 +1,19 @@ -This document outlines setting up the default OPA distribution with Istio.Ultimately, the plug-and-play solution of OPA for Istio was not used however these instructions are provided as reference. +# Setting up the default OPA distribution with Istio -## NOTE: -We ultimately did NOT proceed with the default OPA-Istio distribution. It places a sidecar at each pod (which we still do) however, it also intercepts every call. We would rather intercept the calls at the WASM Plugin and make validation requests to the opa sidecar only if necessary. This default distribution is creatd with either OPA-Filter.yml or quickstart.yml. The difference between these two files is the evaluation policy. +This document outlines setting up the default OPA distribution with Istio. Ultimately, the plug-and-play solution of OPA for Istio was not used however these instructions are provided as reference. + +## NOTE + +We ultimately did NOT proceed with the default OPA-Istio distribution. It places a sidecar at each pod (which we still do) however, it also intercepts every call. We would rather intercept the calls at the WASM Plugin and make validation requests to the opa sidecar only if necessary. This default distribution is created with either OPA-Filter.yml or quickstart.yml. The difference between these two files is the evaluation policy. The default tutorial this document is based off can be found here: -https://www.openpolicyagent.org/docs/latest/envoy-tutorial-istio/ + --- -**We are going to start w/ an empty Minikube Environment** +**We are going to start w/ an empty Minikube Environment** `minikube start` -**Install Istio** +**Install Istio** `istioctl install --set profile=demo -y` `kubectl label namespace default istio-injection=enabled` `istioctl analyze` @@ -18,26 +21,28 @@ https://www.openpolicyagent.org/docs/latest/envoy-tutorial-istio/ **Enable injection for OPA** `kubectl label namespace default opa-istio-injection="enabled"` -**Applying OPA's Resources (files are in /OPA)** +**Applying OPA's Resources (files are in /OPA)** `kubectl delete -f OPA-Filter.yaml` `kubectl apply -f OPA-Filter.yaml` **MAKE SURE TO SET UP TUNNEL IN OTHER TERMINAL** `minikube tunnel` + * Check that the relevant SVC shows an external IP -- `kubectl get service identify-svc-lb` - -# Testing OPA-Filter.yaml Filter on Account-CRUD App -- NOTE: This `curl` command wont' work. Why? -> Filter requires baggage header. Curling app directly means there no baggage enetering the account-CRUD app. -- TLDR: Don't use: `curl $(minikube service flask-service --url)/users` - -- INSTEAD: You MUST hit Identify Svc -> /users Endpoint. Either of these work: -- `curl $(kubectl get svc identify-svc-lb -o jsonpath='{.status.loadBalancer.ingress[0].ip}')/users` -- `curl $(minikube service identify-svc-lb --url)/users` - -- NOTE: The intended behavior is that the response to this request alternates b/w Error 500 & the actual response (user information). The filter currently makes a request to the 'eval-policy-true-svc' to verify a label's access. We alternate b/w labels 1 & 2 (allowed/disallowed) thus the filter must alternate between allowing requests and not. - -- Editing Flow: Of course you'll need to redploy your manifests. You'll also restart the deploy to clear the cache. Then watch the pods to make sure all containers in pod come alive. +* `kubectl get service identify-svc-lb` + +## Testing OPA-Filter.yaml Filter on Account-CRUD App + +* NOTE: This `curl` command wont' work. Why? -> Filter requires baggage header. Curling app directly means there no baggage entering the account-CRUD app. +* TLDR: Don't use: `curl $(minikube service flask-service --url)/users` + +* INSTEAD: You MUST hit Identify Svc -> /users Endpoint. Either of these work: +* `curl $(kubectl get svc identify-svc-lb -o jsonpath='{.status.loadBalancer.ingress[0].ip}')/users` +* `curl $(minikube service identify-svc-lb --url)/users` + +* NOTE: The intended behavior is that the response to this request alternates b/w Error 500 & the actual response (user information). The filter currently makes a request to the 'eval-policy-true-svc' to verify a label's access. We alternate b/w labels 1 & 2 (allowed/disallowed) thus the filter must alternate between allowing requests and not. + +* Editing Flow: Of course you'll need to redploy your manifests. You'll also restart the deploy to clear the cache. Then watch the pods to make sure all containers in pod come alive. `kubectl delete -f OPA-Filter.yaml` `kubectl apply -f OPA-Filter.yaml` `kubectl rollout restart deployment -l app=flaskapi` @@ -45,30 +50,40 @@ https://www.openpolicyagent.org/docs/latest/envoy-tutorial-istio/ `curl $(minikube service identify-svc-lb --url)/users` --- -# Excluding Pods from OPA Filtering -* This is a 2 step Process + +## Excluding Pods from OPA Filtering + +This is a 2 step Process + 1. In the MutatingWebhookConfiguration manifest, on the same level as "namespaceSelector" paste this: -``` yaml - objectSelector: - matchExpressions: - - key: sidecar.opa-istio.io/inject - operator: NotIn - values: - - "false" -``` - -2. In any pod you don't want the OPA filter to apply in, go to their Deployment and add this to their list of labels: -`sidecar.opa-istio.io/inject: "false"` - + + ```yaml + objectSelector: + matchExpressions: + - key: sidecar.opa-istio.io/inject + operator: NotIn + values: + - "false" + ``` + +1. In any pod you don't want the OPA filter to apply in, go to their Deployment and add this to their list of labels: + + ```yaml + sidecar.opa-istio.io/inject: "false" + ``` + --- -# NOTE: -- OPA Filters MUST allow Healthcheck requests otherwise k8s thinks pods aren't deploying correctly -# TIP: -- This is the format for getting the logs of the Opa Proxy -`kubectl logs -c=opa-istio` +> [!NOTE] +> OPA Filters MUST allow Healthcheck requests otherwise k8s thinks pods aren't deploying correctly + +--- + +> [!TIP] +> This is the format for getting the logs of the Opa Proxy: `kubectl logs -c=opa-istio` + +## Caching is a Problem -# Caching is a Problem: OPA Sidecar will Cache Evaluations EVEN IF you delete the yaml. You have to restart the sidecar as well! To do this for the Account CRUD App: `kubectl rollout restart deployment -l app=flaskapi` diff --git a/docs/OpenTelemetryBasics.md b/docs/OpenTelemetryBasics.md index 992f7b7..49e23af 100644 --- a/docs/OpenTelemetryBasics.md +++ b/docs/OpenTelemetryBasics.md @@ -1,44 +1,36 @@ # Open Telemetry Basics -Here we provide resources & code snippets about Open Telemetry. All of the basics learned here were applied and consildated into a handful of easy to use functions in `lineage_propagation.py`. + +Here we provide resources & code snippets about OpenTelemetry. All of the basics learned here were applied and consolidated into a handful of easy to use functions in `lineage_propagation.py`. --- -I have found Aspecto's articles on OpenTelemetry to be the most helpful. Here are two: -1. https://www.aspecto.io/blog/distributed-tracing-with-opentelemetry-collector-on-kubernetes/ -2. https://www.aspecto.io/blog/opentelemetry-collector-agent-on-kubernetes/ +The following articles on OpenTelemetry are the most helpful for what we want to +accomplish: -Lightstep has also been very useful, particularly this article/repo: -1. https://lightstep.com/blog/opentelemetry-for-python-the-hard-way -2. https://github.com/lightstep/opentelemetry-examples/tree/main/python/opentelemetry/manual_instrumentation +1. +1. +1. +1. ---- -### NOTE -I use Aspecto's visualization tools instead of Jaeger's as they are a bit more intuitive for me. Doing so requires an Aspecto API key and thus an Aspecto user account. --- -# Deploying OTel Resources -Included here & in the provided files is reference to and instructions for exporting traces to Aspecto. You must have an account there & retrieve your API key to use Aspecto's visualization. +## Deploying OTel Resources + +Included here & in the provided files is reference to and instructions for exporting traces to Jaeger. * Deploy Gateway for Otel (Creates opentelemetry namespace) `kubectl apply -f gateway.yml` -* Create Secret of Aspecto API Key in opentelemetry namespace -- Switch Namespace to opentelemetry +* Switch Namespace to opentelemetry `kubectl config set-context --current --namespace=opentelemetry` -* Create secret -`kubectl create secret generic aspecto --from-literal=api-key=` - -* Verify Secret Created -`kubectl get secrets -o json` - * Switch namespace to default `kubectl config set-context --current --namespace=default` - + * Deploy ConfigMap for Sidecar `kubectl apply -f otel-agent-config.yml` -* Deploy Application and Sidecar: +* Deploy Application and Sidecar: `kubectl apply -f .yml` * Deploy Jaeger all in one to consume Otel data @@ -50,12 +42,11 @@ Included here & in the provided files is reference to and instructions for expor * Jaeger url `localhost:16686/search` -* Aspecto -https://www.aspecto.io/ - --- -# Instrumenting Flask -* Exporter Docs Link: https://opentelemetry.io/docs/instrumentation/python/exporters/#using-http + +## Instrumenting Flask + +* Exporter Docs Link: ``` python #Imports @@ -83,16 +74,18 @@ provider.add_span_processor(processor) trace.set_tracer_provider(provider) #Creating a span - #Get Tracer from Trace - tracer = trace.get_tracer(__name__) - - #Define Span - with tracer.start_as_current_span("rootSpan"): - #Indent code in span below + #Get Tracer from Trace + tracer = trace.get_tracer(__name__) + + #Define Span + with tracer.start_as_current_span("rootSpan"): + #Indent code in span below ``` --- -# Propagating Trace + +## Propagating Trace + ``` python from flask import request from opentelemetry import context @@ -122,7 +115,7 @@ if traceparent: # `with tracer.start_as_current_span("rootSpan"):` #Necessary if Propagating Downwards -#Propogating Context +#Propagating Context carrier = {} TraceContextTextMapPropagator().inject(carrier) header = {"traceparent": carrier["traceparent"]} @@ -139,7 +132,8 @@ header = {"traceparent": carrier["traceparent"]} ``` --- -# Baggage Creation & Propagation + +## Baggage Creation & Propagation ``` python #Creating Baggage @@ -199,11 +193,13 @@ detach(token) ``` * NOTE: If you don't want to actually log traces out to otlp and instead just use the traces do the following: + 1. Comment out Otel Collector Agent from deployment files and redeploy 2. Remove deployment of Gateway `kubectl delete -f gateway.yml` 3. Remove Jaeger `kubectl delete -f jaeger.yml` 4. Remove/Comment out BatchSpanProcessor (found in python) -- processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")) -- provider.add_span_processor(processor) + +* processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="")) +* provider.add_span_processor(processor) diff --git a/docs/UserAccountCRUD.md b/docs/UserAccountCRUD.md index ee7e5ca..82c6ab1 100644 --- a/docs/UserAccountCRUD.md +++ b/docs/UserAccountCRUD.md @@ -1,28 +1,38 @@ +# User Account CRUD + Based On: -https://www.kdnuggets.com/2021/02/deploy-flask-api-kubernetes-connect-micro-services.html + + --- ## Editing the CRUD + ### After Editing the .py Flask file + `docker build . -t /flask-user-accounts-api` `kubectl delete -f flaskapp-deployment.yml` `kubectl apply -f flaskapp-deployment.yml` #### Get API access URL + `minikube service flask-service --url` ### Modifying the Database + - Add SQL instruction to ConfigMap section in mysql Deployment file ### After Editing any of the mysql Manifest Files OR if you'd like to reset the database + `kubectl delete -f mysql-pv.yml` `kubectl apply -f mysql-pv.yml` `kubectl delete -f mysql-deployment.yml` `kubectl apply -f mysql-deployment.yml` -#### For Reference: +#### For Reference + ##### Creating Mysql Pod With Preloaded Database -- https://medium.com/@AbhijeetKasurde/creating-mysql-pod-with-preloaded-database-2c01c002fdc3 + +- --- If K8s was Deleted, you must apply various yml configurations @@ -31,4 +41,4 @@ Applying Configurations `kubectl apply -f flaskapi-secrets.yml` `kubectl apply -f mysql-pv.yml` `kubectl apply -f mysql-deployment.yml` -`kubectl apply -f flaskapp-deployment.yml` \ No newline at end of file +`kubectl apply -f flaskapp-deployment.yml` diff --git a/docs/Using-lineage_propagation.py.md b/docs/Using-lineage_propagation.py.md index 5d5a39c..f8c5112 100644 --- a/docs/Using-lineage_propagation.py.md +++ b/docs/Using-lineage_propagation.py.md @@ -11,52 +11,55 @@ from lineage_propagation import * # exporting_traces: Boolean - toggles trace exporting # export_endpoint: str - provide url of Otel Collector Agent(defaulted to Otel default) initialize_tracing() - + #Enable Tracing on a single Endpoint - #Beneath app.route("/") - #If no string is provided, will use default string - @enable_tracing("call-endpt-svc-V2.3") - - #Add Parent Trace information to Header and Returns updated Header - #Does not require existing headers to be passed in - headers = add_tracing_header(headers) - - #Adds passed in Lineage Label to headers and Returns: - #updated Header - #Token that can be used to undo Label Append - #Does not require existing headers to be passed in - headers, linCtxToken = add_lineage_label("call-endpt-v2.3", headers) - - #Undo changes related to Token passed in - #Must pass in token - undo_label_change(linCtxToken) + #Beneath app.route("/") + #If no string is provided, will use default string + @enable_tracing("call-endpt-svc-V2.3") + + #Add Parent Trace information to Header and Returns updated Header + #Does not require existing headers to be passed in + headers = add_tracing_header(headers) + + #Adds passed in Lineage Label to headers and Returns: + #updated Header + #Token that can be used to undo Label Append + #Does not require existing headers to be passed in + headers, linCtxToken = add_lineage_label("call-endpt-v2.3", headers) + + #Undo changes related to Token passed in + #Must pass in token + undo_label_change(linCtxToken) - #Get the current Label Set - my_var = get_lineage_label_set() - - #Override Label Lineage to Passed in String - #Does not require the existing headers to be passed in - headers, linCtxToken = set_lineage_label_set("call-endpt-WARNING", headers) - - #Adds Lineage Label Set to Span Attributes - add_lineage_to_attributes() - - - #REMINDER: You MUST Pass in headers to next response - #Pass headers in request - response = requests.get(endpoint, headers=headers) + #Get the current Label Set + my_var = get_lineage_label_set() + + #Override Label Lineage to Passed in String + #Does not require the existing headers to be passed in + headers, linCtxToken = set_lineage_label_set("call-endpt-WARNING", headers) + + #Adds Lineage Label Set to Span Attributes + add_lineage_to_attributes() + + + #REMINDER: You MUST Pass in headers to next response + #Pass headers in request + response = requests.get(endpoint, headers=headers) ``` --- + * NOTES -- Your final endpoint does NOT require you to "add_tracing_header" -- Your final endpoint DOES require you to "enable_tracing" in order to capture all propagated baggage -- You CAN add Lineage Labels at the final endpoint -- You CANNOT propagate attributes +* Your final endpoint does NOT require you to "add_tracing_header" +* Your final endpoint DOES require you to "enable_tracing" in order to capture all propagated baggage +* You CAN add Lineage Labels at the final endpoint +* You CANNOT propagate attributes --- + # Adding Tracing to an existing Service: Beyond the Instrumentation + Beyond the instrumentation of the service itself, if you want the OTel visualizations, you'll have to ensure that you have an OTel container: ``` yaml @@ -78,6 +81,7 @@ Beyond the instrumentation of the service itself, if you want the OTel visualiza ``` Make sure to include these environment variables in your app's container otherwise Otel won't export + ```yaml - name: SERVICE_NAME value: @@ -85,4 +89,4 @@ Make sure to include these environment variables in your app's container otherwi value: ``` -See the articles provided in `OpenTelemetryBasics` for more info about manifest files. \ No newline at end of file +See the articles provided in `OpenTelemetryBasics` for more info about manifest files. diff --git a/docs/VSCode Server.md b/docs/VSCode Server.md index c5e13d1..963c829 100644 --- a/docs/VSCode Server.md +++ b/docs/VSCode Server.md @@ -1,21 +1,24 @@ -# Quick Start +# VS Code Server + +## Quick Start + * In Terminal create tunnel then launch server: -- ssh -L 8080:localhost:8080 -- code-server --auth none +* `ssh -L 8080:localhost:8080 ` +* code-server --auth none * In Browser: -- Go To localhost:8080 - +* Go To localhost:8080 + --- -# Full Instructions +## Full Instructions -## Install VSCode Server +### Install VSCode Server In order to remotely edit files on the EC2 instance using Visual Studio Code, you'll need to install the Visual Studio Code server on the EC2 instance. The Visual Studio Code server allows you to remotely access the Visual Studio Code editor from your local machine. - + * To install the Visual Studio Code server on the EC2 instance, you'll need to follow these steps: -- Connect to the EC2 instance using SSH: `ssh -L 8080:localhost:8080 ` -- Install the Visual Studio Code server by running the following command: `curl -fsSL https://code-server.dev/install.sh | sh` -- Once the installation is complete, start the server by running the following command: - `code-server --auth none` \ No newline at end of file +* Connect to the EC2 instance using SSH: `ssh -L 8080:localhost:8080 ` +* Install the Visual Studio Code server by running the following command: `curl -fsSL https://code-server.dev/install.sh | sh` +* Once the installation is complete, start the server by running the following command: + `code-server --auth none` diff --git a/docs/WASM-Label-Lineage.md b/docs/WASM-Label-Lineage.md index d859abe..8db32b6 100644 --- a/docs/WASM-Label-Lineage.md +++ b/docs/WASM-Label-Lineage.md @@ -1,23 +1,28 @@ -# Relevant Files: +# Relevant Files + * Everything in account-CRUD: This is your Test app/service chain * Istio-1.6.1/wasm-lineage-headers: This is your actual plugin --- The following deployment strategy and relevant EnvoyFilter were created referencing this: -https://github.com/halfrost/sentinel-go-envoy-proxy-wasm + --- - -# Before: + +## Before + We were deploying w/ wasm.yml & using the Istio WasmPlugin resource. However. b/c we need to be able to set the vm_id, we have to deploy w/ EnvoyFilter. -- The Files named "envoyFilter" and "envoyFilter2" both deploy using a "local" .wasm binary file. These can found in the archive folder. The file is local to the flaskapp & uploaded w/ a configmap: `kubectl create configmap lin-hdrs-filter --from-file=lin-hdrs-filter.wasm=main.wasm` -- We DON’T use the Local Deployment Option anymore. -- Instead, we deploy w/ EnvoyFilterRemote. This retrieves the filter from a remote repo (GC storage). We do this so that the manifest of the service we would like to apply the filter to does not need to be edited. - -# Now: + +* The Files named "envoyFilter" and "envoyFilter2" both deploy using a "local" .wasm binary file. These can found in the archive folder. The file is local to the flaskapp & uploaded w/ a configmap: `kubectl create configmap lin-hdrs-filter --from-file=lin-hdrs-filter.wasm=main.wasm` +* We DON’T use the Local Deployment Option anymore. +* Instead, we deploy w/ EnvoyFilterRemote. This retrieves the filter from a remote repo (GC storage). We do this so that the manifest of the service we would like to apply the filter to does not need to be edited. + +## Now + These are the steps to follow after editing the main.go file: *after editing your main.go* + 1. Build the go file into a wasm binary `tinygo build -o main.wasm -scheduler=none -target=wasi main.go` 2. Update the yaml to contain the right sha256 hash @@ -30,38 +35,41 @@ These are the steps to follow after editing the main.go file: 4. Redploy Envoy Filter `kubectl delete -f envoyFilterRemote.yaml` `kubectl apply -f envoyFilterRemote.yaml` - + * View the Filter's Logs `kubectl logs $(kubectl get pod -l app=flaskapi -o name) -c istio-proxy` * Curl the service the filter is applied to (and you can view the logs again) `curl $(kubectl get svc identify-svc-lb -o jsonpath='{.status.loadBalancer.ingress[0].ip}')/users` - -# Futher NOTES: -- We Commented out/removed the Config File annotations in the flaskapi & mysql manifests b/c we aren't using the local deployment anymore - -- If you're having a caching problem w/ Istio, append "?wtv=123" to the end of the uri in the deployment file and it will get the new one for sure. It should get un-cached in 1 hour - -- Gets hash of file: `sha256sum main.wasm` - -- There are various GoogleCloud URL formats for bucket objects. This is the only format of URL that works w/ EnvoyFilter: `http://wasm-lineage-headers.storage.googleapis.com/main.wasm?id=1` - -- You HAVE TO do the no-cache metadata thing otherwise Istio will cache it and you can edit & update the filter (You can always do the url thing w/ ?wtv=123) - + +## Futher NOTES + +* We Commented out/removed the Config File annotations in the flaskapi & mysql manifests b/c we aren't using the local deployment anymore + +* If you're having a caching problem w/ Istio, append "?wtv=123" to the end of the uri in the deployment file and it will get the new one for sure. It should get un-cached in 1 hour + +* Gets hash of file: `sha256sum main.wasm` + +* There are various GoogleCloud URL formats for bucket objects. This is the only format of URL that works w/ EnvoyFilter: `http://wasm-lineage-headers.storage.googleapis.com/main.wasm?id=1` + +* You HAVE TO do the no-cache metadata thing otherwise Istio will cache it and you can edit & update the filter (You can always do the url thing w/ ?wtv=123) + --- Other Things we learned: + * vm_id has to be the same for various plugins that want to access the same data - + * This list of Packages supported by TinyGo is not Exhaustive: -- https://tinygo.org/docs/reference/lang-support/stdlib/#fmt - -# Links -A set of usefule resources - -* Alternative EnvoyFilter Manifest Example (did not use): https://github.com/istio-ecosystem/wasm-extensions/blob/master/example/config/example-filter.yaml - -* Best Explanation for Threads and Wasm Modules: https://thenewstack.io/wasm-modules-and-envoy-extensibility-explained-part-1/ - -* Go Wasm SDK Documentation: https://github.com/tetratelabs/proxy-wasm-go-sdk/blob/main/doc/OVERVIEW.md - -* Gjson - Easiest way to get JSON working in TinyGo: https://github.com/tidwall/gjson +* + +## Links + +A set of useful resources + +* Alternative EnvoyFilter Manifest Example (did not use): + +* Best Explanation for Threads and Wasm Modules: + +* Go Wasm SDK Documentation: + +* Gjson - Easiest way to get JSON working in TinyGo: diff --git a/docs/WASMPluginFromScratch.md b/docs/WASMPluginFromScratch.md index 5384eab..b016a48 100644 --- a/docs/WASMPluginFromScratch.md +++ b/docs/WASMPluginFromScratch.md @@ -1,21 +1,24 @@ -This is only a general overview for writing WASM plugins from scratch. -- NOTE: All dependencies must already be installed (Go, TinyGo, Minikube, Istio, docker) +# WASM Plugin From Scratch -`mkdir && cd ` +This is only a general overview for writing WASM plugins from scratch. -**Initialize the Go Module** -`go mod init ` +> [!NOTE] +>All dependencies must already be installed (Go, TinyGo, Minikube, Istio, docker) -*write main.go file* +`mkdir && cd ` -**Download the dependencies** -`go mod tidy` +- Initialize the Go Module** + `go mod init ` +- write main.go file* +- Download the dependencies** + `go mod tidy` +- Write the yaml manifest file** -**Write the yaml manifest file** - We can use either a WasmPlugin resource or an EnvoyFilter. WasmPlugin is easier and more abstract but lacks a number of features which is why the demo account-CRUD uses EnvoyFilter **Compile the Wasm Plugin** `rm main.wasm` + - seemingly necessary since the following the command doesn't override existing file `tinygo build -o main.wasm -scheduler=none -target=wasi main.go` @@ -37,4 +40,4 @@ The steps prior to this vary depending on whether you are deploying using a remo The above CLI commands for creating & deploying a WASM plugin do so using GoogleCloud to store the WASM file for remote access and deploy it with a EnvoyFilter resource. We deploy this way so that the application we are applying this filter to does not need its manifest modified. -(See DeployEnvoyFilterIstio for Deploying using local file). \ No newline at end of file +(See DeployEnvoyFilterIstio for Deploying using local file). diff --git a/docs/WritingOPAPolicies.md b/docs/WritingOPAPolicies.md index ab70c9e..9c282c8 100644 --- a/docs/WritingOPAPolicies.md +++ b/docs/WritingOPAPolicies.md @@ -3,12 +3,14 @@ The playground allows you to test a lot of things BUT some functions don't work Solution: Install OPA CLI & Test Policies/Data locally -**Installing OPA CLI** -https://sangkeon.github.io/opaguide/chap2/installandusage.html ---- +**Installing OPA CLI** + +--- + **My Test Files:** `cd OPA/wasm-call-test` -- This contains 2 files. 1 Rego Policy file. 1 file w/ data for the policy. Each of these are .rego files. They can freely be edited and tested. Reference this https://www.openpolicyagent.org/docs/latest/policy-testing/ for test formatting. Essentially, your rego to evaluate is exactly as normal. The other file must include a package name, imports, and the functions must be prefixed w/ "test_". Each Test (yes you can do multiple) is started w/ something like "allow with input as". This checks if the policy you wrote in the other file "allows" the following input. - + +- This contains 2 files. 1 Rego Policy file. 1 file w/ data for the policy. Each of these are .rego files. They can freely be edited and tested. Reference this for test formatting. Essentially, your rego to evaluate is exactly as normal. The other file must include a package name, imports, and the functions must be prefixed w/ "test_". Each Test (yes you can do multiple) is started w/ something like "allow with input as". This checks if the policy you wrote in the other file "allows" the following input. + **Run the OPA CLI Test command** -`opa test . -v` \ No newline at end of file +`opa test . -v` diff --git a/mysqldb/REAME.md b/mysqldb/REAME.md index 5202e11..0362217 100644 --- a/mysqldb/REAME.md +++ b/mysqldb/REAME.md @@ -1,17 +1,23 @@ # MySQL Database + ## 1. Description + The database of choice within this project is MySQL, which will be running as a standalone service within the cluster. Therefore, this directory will contain all the required files to configure and deploy the MySQL service. ## 2. Deployment + In order to ensure the good functioning of this component service the following steps should be undertaken in the specified order: - - kubectl apply -f _./mysql-secrets.yml_ - - kubectl apply -f _./mysql-persistenvolume.yml_ - - kubectl apply -f _./mysql-persistenvolumeclaim.yml_ - - kubectl apply -f _./mysql-deployment.yml_ - - kubectl apply -f _./mysql-configmap.yml_ - - kubectl apply -f _./mysql-service.yml_ + +- kubectl apply -f _./mysql-secrets.yml_ +- kubectl apply -f _./mysql-persistenvolume.yml_ +- kubectl apply -f _./mysql-persistenvolumeclaim.yml_ +- kubectl apply -f _./mysql-deployment.yml_ +- kubectl apply -f _./mysql-configmap.yml_ +- kubectl apply -f _./mysql-service.yml_ ## 2. Automated Service Start + This service can be brought up automatically by running: + - ./start.sh diff --git a/wasm-lineage-headers/README.md b/wasm-lineage-headers/README.md index 2bb68bd..4650c84 100644 --- a/wasm-lineage-headers/README.md +++ b/wasm-lineage-headers/README.md @@ -2,24 +2,30 @@ This directory contains all files relevant to the development of the tinygo plugin written for the Istio sidecar which parses, validates & caches LabelSets. -## 'main.go' +## 'main.go' + This is the plugin written using tinygo and the go proxywasm library. ## 'main_no_opa.go' + A slightly modified version of the plugin that removes and instead mocks the OPA call but retains the Caching mechanism of the plugin. This was used to test the overhead of OPA. -## 'envoyFilterRemote.yaml' +## 'envoyFilterRemote.yaml' + One part of the process of deploying the plugin. Notably, it pulls in the plugin in remotely from GoogleCloud as opposed to loading the plugin from a local file in a ConfigMap. See other documentation for local ConfigMap details. We deploy this way so that the application we are applying this filter to does not need its manifest modified. -## 'update_yaml.sh' +## 'update_yaml.sh' + A helper script used in deployment that helpfully changes the hash in the 'envoyFilterRemote.yaml' file so that Istio actually changes out the cached plugin instantly instead of needing to wait an hour ## 'wasm.yml' (deprecated) + A manifest file for Istio's WasmPlugin resource. However, their implementation lack some configuration options when compared to the less abstract EnvoyFilter (see note below), so an EnvoyFilter manifest was used instead. Note: Specifically, we desired the ability to set a vm_id so that multiple plugins can share access to the same cache. -## 'utl-wasm-lin-headers.txt' +## 'utl-wasm-lin-headers.txt' + Contains a number of variuos helpful commands for dealing with this directory. Of note, there is the list of "#UPDATED - FINAL Editing FLOW" commands which (after proper configuration of GoogleCloud) can just be pasted and will properly deploy the updated plugin. (Though make sure to watch the console as errors for the plugin can be difficult to see through all the text. Of course, errors mean the new plugin won't deploy.) From f581472a7bf6537c1346d20043db94c18023dabc Mon Sep 17 00:00:00 2001 From: svrnm Date: Wed, 19 Jun 2024 10:20:57 +0200 Subject: [PATCH 2/3] remove trailing space Signed-off-by: svrnm --- docs/OpenTelemetryBasics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/OpenTelemetryBasics.md b/docs/OpenTelemetryBasics.md index 49e23af..b7a3197 100644 --- a/docs/OpenTelemetryBasics.md +++ b/docs/OpenTelemetryBasics.md @@ -202,4 +202,4 @@ detach(token) 4. Remove/Comment out BatchSpanProcessor (found in python) * processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="")) -* provider.add_span_processor(processor) +* provider.add_span_processor(processor) \ No newline at end of file From 67aad75978d72d89ba9fb4a8d7aaec9238a5349f Mon Sep 17 00:00:00 2001 From: svrnm Date: Wed, 19 Jun 2024 10:24:14 +0200 Subject: [PATCH 3/3] fix trailing space and eof Signed-off-by: svrnm --- docs/OpenTelemetryBasics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/OpenTelemetryBasics.md b/docs/OpenTelemetryBasics.md index b7a3197..595c861 100644 --- a/docs/OpenTelemetryBasics.md +++ b/docs/OpenTelemetryBasics.md @@ -16,7 +16,7 @@ accomplish: ## Deploying OTel Resources -Included here & in the provided files is reference to and instructions for exporting traces to Jaeger. +Included here & in the provided files is reference to and instructions for exporting traces to Jaeger. * Deploy Gateway for Otel (Creates opentelemetry namespace) `kubectl apply -f gateway.yml` @@ -202,4 +202,4 @@ detach(token) 4. Remove/Comment out BatchSpanProcessor (found in python) * processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="")) -* provider.add_span_processor(processor) \ No newline at end of file +* provider.add_span_processor(processor)