-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
174 changed files
with
34,682 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
import json | ||
from opentelemetry import trace | ||
|
||
tracer = trace.get_tracer(__name__) | ||
|
||
def lambda_handler(event, context): | ||
with tracer.start_as_current_span("lambda_handler"): | ||
# Your business logic here | ||
return { | ||
'statusCode': 200, | ||
'body': json.dumps('Hello from Middleware!') | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
# Python APM Guide | ||
You can follow our [documentation](https://docs.middleware.io/docs/apm-configuration/python/python-apm-setup) to setup APM for your Python application. | ||
|
||
[![PyPI - Version](https://img.shields.io/pypi/v/middleware-apm)](https://pypi.org/project/middleware-apm/) | ||
|
||
|
||
| Traces | Metrics | Profiling | Logs (App/Custom) | | ||
|:--------:|:---------:|:-----------:|:-------------------:| | ||
| Yes | Yes | Yes | Yes/Yes | | ||
|
||
## Prerequisites | ||
Ensure that you have the Middleware Host Agent installed to view Python demo data on your dashboard. | ||
|
||
--------------------- | ||
|
||
## Installing the Package | ||
Run the following commands in your terminal: | ||
### Step 1: Install Middleware APM package | ||
```shell | ||
pip install middleware-apm | ||
``` | ||
|
||
## Your Sample Code | ||
By using all the MW's APM functionalities like: Distributed-tracing, Logs, Metrics and Profiling, your code will look like this: | ||
```python | ||
import logging | ||
|
||
from middleware import MwTracker | ||
tracker=MwTracker() | ||
|
||
logging.warning("Sample Warning Log.") | ||
logging.error("Sample Error Log.", extra={'tester': 'Alex'}) | ||
``` | ||
## Setup middleware.ini File | ||
Setup your `middleware.ini` file, based on below features that you want to observe in your project. Place the file at the root of your project. | ||
```ini | ||
# --------------------------------------------------------------------------- | ||
# This file contains settings for the Middleware Python-APM Agent. | ||
# Here are the settings that are common to all environments. | ||
# --------------------------------------------------------------------------- | ||
|
||
[middleware.common] | ||
|
||
# The name of your application as service-name, as it will appear in the UI to filter out your data. | ||
service_name = Python-APM-Service | ||
|
||
# This Token binds the Python Agent's data and profiling data to your account. | ||
access_token = {YOUR-ACCESS-TOKEN} | ||
|
||
# The service name, where Middleware Agent is running, in case of K8s. | ||
;mw_agent_service = mw-service.mw-agent-ns.svc.cluster.local | ||
|
||
# Toggle to enable/disable distributed traces for your application. | ||
collect_traces = true | ||
|
||
# Toggle to enable/disable the collection of metrics for your application. | ||
collect_metrics = true | ||
|
||
# Toggle to enable/disable the collection of logs for your application. | ||
collect_logs = true | ||
|
||
# Toggle to enable/disable the collection of profiling data for your application. | ||
collect_profiling = true | ||
|
||
# --------------------------------------------------------------------------- | ||
``` | ||
#### Note: You need to replace <strong>\{YOUR-ACCESS-TOKEN\}</strong> with your APM's Access Token. | ||
|
||
|
||
## Setup Virtual Environment | ||
``` | ||
python -m venv newenv | ||
``` | ||
``` | ||
source newenv/bin/activate | ||
``` | ||
``` | ||
pip install -r requirements.txt | ||
``` | ||
|
||
## Run Your Application | ||
|
||
### Option 1 : With Host Agent | ||
To run your application, use the following command: | ||
```shell | ||
middleware-apm run python app.py | ||
``` | ||
### Option 2 : Serverless Setup | ||
```shell | ||
MW_API_KEY=********** MW_TARGET=https://*****.middleware.io:443 middleware-apm run python app.py | ||
``` | ||
#### Note: If `middleware.ini` isn't in your project's root, set `MIDDLEWARE_CONFIG_FILE=./path/to/middleware.ini` along with the above run command. | ||
|
||
--------------------------------- | ||
|
||
## Troubleshooting Demo | ||
If you face any protoc specific errors, Try setting ... | ||
``` | ||
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python | ||
``` | ||
--------------------------------- | ||
## Run on Docker | ||
1. Build: `docker build -t demo-python .` | ||
2. Run: `docker run demo-python` | ||
3. Debug: `docker run -it demo-python sh` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,116 @@ | ||
from flask import Flask, jsonify, request | ||
import traceback | ||
import logging | ||
from functools import wraps | ||
from opentelemetry import trace, metrics | ||
from opentelemetry.trace.status import StatusCode | ||
|
||
|
||
from middleware import MwTracker | ||
tracker=MwTracker() | ||
|
||
# Initialize Flask app | ||
app = Flask(__name__) | ||
|
||
# Trace provider | ||
tracer = trace.get_tracer("custom-tracer") | ||
|
||
# Metric provider | ||
meter = metrics.get_meter("custom-meter") | ||
|
||
# Setup Python logging | ||
logging.basicConfig(level=logging.INFO) | ||
logger = logging.getLogger(__name__) | ||
|
||
# Custom Metrics: Counter | ||
request_counter = meter.create_counter( | ||
name="requests_total", | ||
description="Total number of requests", | ||
unit="1" | ||
) | ||
|
||
# Custom Traces: Tracing decorator with error handling | ||
def trace_request(func): | ||
@wraps(func) # Preserve the original function's name and docstring | ||
def wrapper(*args, **kwargs): | ||
with tracer.start_as_current_span(func.__name__) as span: | ||
span.set_attribute("custom.attribute", "example_value") | ||
|
||
# Log the trace and span IDs | ||
span_context = span.get_span_context() | ||
logger.info(f"Start processing {func.__name__}: trace_id={span_context.trace_id}, span_id={span_context.span_id}") | ||
|
||
try: | ||
result = func(*args, **kwargs) | ||
|
||
# Only check status code for POST request handling | ||
if request.method == 'POST': | ||
if result.status_code >= 400: | ||
span.set_status(StatusCode.ERROR) | ||
logger.error(f"Error occurred in {func.__name__}: status_code={result.status_code}") | ||
|
||
return result | ||
|
||
except Exception as e: | ||
span.set_status(StatusCode.ERROR) | ||
span.record_exception(e) | ||
logger.error(f"Exception occurred in {func.__name__}: {str(e)}") | ||
logger.error(f"Stack Trace:\n{traceback.format_exc()}") # Log the full stack trace | ||
return jsonify({"error": "An internal error occurred"}), 500 | ||
return wrapper | ||
|
||
@app.route('/') | ||
@trace_request | ||
def home(): | ||
request_counter.add(1, {"endpoint": "home"}) | ||
logger.info("Home endpoint accessed") | ||
return jsonify({"message": "Welcome to the Flask app!"}) | ||
|
||
@app.route('/process', methods=['GET', 'POST']) | ||
@trace_request | ||
def process_data(): | ||
if request.method == 'GET': | ||
# Render a simple HTML form for demonstration | ||
return ''' | ||
<form method="POST"> | ||
<label for="data">Enter some data (JSON format):</label><br> | ||
<textarea id="data" name="data" rows="4" cols="50"></textarea><br> | ||
<input type="submit" value="Submit"> | ||
</form> | ||
''' | ||
|
||
if request.method == 'POST': | ||
# Process JSON data submitted via the form | ||
try: | ||
data = request.json if request.is_json else request.form.get('data') | ||
request_counter.add(1, {"endpoint": "process"}) | ||
logger.info(f"Processing data: {data}") | ||
|
||
with tracer.start_as_current_span("data_processing") as span: | ||
span.set_attribute("request.data", str(data)) | ||
# Simulate processing | ||
processed_data = {"processed": data} | ||
logger.info("Data processed successfully") | ||
|
||
response = jsonify(processed_data) # Create response object | ||
|
||
return response # Return the response object | ||
|
||
except Exception as e: | ||
span.set_status(StatusCode.ERROR) | ||
span.record_exception(e) | ||
logger.error(f"Exception occurred in process_data: {str(e)}") | ||
logger.error(f"Stack Trace:\n{traceback.format_exc()}") | ||
return jsonify({"error": "An internal error occurred"}), 500 | ||
|
||
@app.route('/error') | ||
@trace_request | ||
def error(): | ||
request_counter.add(1, {"endpoint": "error"}) | ||
logger.warning("Error endpoint accessed, simulating an error") | ||
|
||
# Simulate an exception to trigger stack trace logging | ||
raise ValueError("Simulated internal server error") | ||
|
||
if __name__ == '__main__': | ||
app.run(port=5000) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,156 @@ | ||
# --------------------------------------------------------------------------- | ||
# This file contains settings for the Middleware Python-APM Agent. | ||
# some settings are commented out to use uncomment as per requirement. | ||
# Add middleware.ini to root folder or set env MIDDLEWARE_CONFIG_FILE with file path. | ||
# Note: Envs can not be used in config file middleware.ini below are shown for reference only. | ||
# --------------------------------------------------------------------------- | ||
|
||
# Here are the settings that are common to all environments. | ||
[middleware.common] | ||
|
||
# Desc: The name of your application as service-name, as it will appear in the UI to filter out your data. | ||
# Env: MW_SERVICE_NAME or OTEL_SERVICE_NAME | ||
# Type: String | ||
# Required (uses 'service-pid' if not declared ) | ||
|
||
service_name = python-apm-service-noinstrument | ||
|
||
|
||
|
||
# Desc: This Token binds the Python Agent's data and required for profiling data and serverless usage. | ||
# Env: MW_API_KEY | ||
# Type: String (abcdefghijklmnopqrstuvwxyz) | ||
# Required | ||
|
||
access_token = {replace-access-token} | ||
|
||
|
||
|
||
# Desc: Toggle to enable/disable traces for your application. | ||
# Env: MW_APM_COLLECT_TRACES | ||
# Type: Bool | ||
# Optional (Default True) | ||
|
||
collect_traces = true | ||
|
||
|
||
|
||
# Desc:Toggle to enable/disable the collection of metrics for your application. | ||
# Env: MW_APM_COLLECT_METRICS | ||
# Type: Bool | ||
# Optional (Default False) | ||
|
||
collect_metrics = true | ||
|
||
|
||
|
||
# Desc: Toggle to enable/disable the collection of logs for your application. | ||
# Env: MW_APM_COLLECT_LOGS | ||
# Type: Bool | ||
# Optional (Default True) | ||
|
||
collect_logs = true | ||
|
||
|
||
|
||
# Desc: Toggle to enable/disable the collection of profiling data for your application. | ||
# Env: MW_APM_COLLECT_PROFILING | ||
# Type: Bool | ||
# Optional (Default False) | ||
|
||
collect_profiling = true | ||
|
||
|
||
|
||
# Desc: To capture logs based on level change logging log_level. Default is FATAL. | ||
# Env: MW_LOG_LEVEL or OTEL_LOG_LEVEL | ||
# Type: String ( DEBUG, INFO, WARNING, ERROR, CRITICAL, FATAL) | ||
# Optional (Default INFO) | ||
|
||
; log_level= ERROR | ||
|
||
|
||
|
||
# --------------For Host based only(with agent)--------------------- | ||
# Desc: Need to set with agent for K8s or docker based application. | ||
# Should not be used for serveless. Either use "mw_target" or "mw_agent_service". | ||
# Env: MW_AGENT_SERVICE | ||
# Type: String | ||
# Optional (Default localhost) | ||
|
||
# Sample for K8s application | ||
|
||
; mw_agent_service = mw-service.mw-agent-ns.svc.cluster.local | ||
|
||
# Sample for Docker Container, it is IP address of the gateway. | ||
|
||
; mw_agent_service = 172.17.0.1 | ||
|
||
|
||
|
||
# ------------- For Serverless only(without agent)----------------------- | ||
# Desc: Set target if instrumentation is required without installing agent.(change "myuid" with project id found in app site). | ||
# Env: MW_TARGET or OTEL_EXPORTER_OTLP_ENDPOINT | ||
# Type: String | ||
# Optional | ||
|
||
; target = https://myuid.middleware.io:443 | ||
|
||
|
||
|
||
# Desc: To set custom resource attributes for traces, metrics and logs. | ||
# Env: MW_CUSTOM_RESOURCE_ATTRIBUTES | ||
# Type: String(key1=value1,key2=value2) | ||
# Optional | ||
# Sample to add key1 with value1 and so on. | ||
|
||
; custom_resource_attributes = 'key1=value1,key2=value2' | ||
|
||
|
||
|
||
# Desc: To enable and change Context Propagators, default B3 is used. | ||
# Env: MW_PROPAGATORS or OTEL_PROPAGATORS | ||
# Type: String | ||
# Optional | ||
|
||
; otel_propagators = b3 | ||
|
||
|
||
# ------------- For Debugging & Troubleshooting----------------------- | ||
# Desc: Disable Information for APM. By default is false. | ||
# Env: MW_DISABLE_INFO | ||
# Type: Bool | ||
# Optional (Default false) | ||
|
||
; disable_info = true | ||
|
||
|
||
|
||
# ------------- For Debugging & Troubleshooting----------------------- | ||
# Desc: Enable exporting traces, metrics, and logs | ||
# in console. | ||
# Env: MW_CONSOLE_EXPORTER | ||
# Type: Bool | ||
# Optional (Default false) | ||
|
||
; console_exporter = true | ||
|
||
|
||
|
||
# ------------- For Debugging & Troubleshooting----------------------- | ||
# Desc: To Save console exported metrics, traces, and logs to separate files. | ||
# work only if console_exporter is enabled. | ||
# Env: MW_DEBUG_LOG_FILE | ||
# Type: Bool | ||
# Optional (Default false) | ||
|
||
; debug_log_file = true | ||
|
||
|
||
|
||
# Desc: To give a project name to the service. | ||
# Env: MW_PROJECT_NAME | ||
# Type: String | ||
# Optional | ||
|
||
; project_name = my-python-project |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
middleware-apm==1.2.1 | ||
flask==2.3.2 |
Oops, something went wrong.