Tracer Linux Agent: Observability for Scientific HPC Workloads

What Is Tracer and Why Use It?

• Tracer is a system-level monitoring platform purpose-built for scientific computing. It is a a one-line install Linux agent and instant dashboards to give you insights into pipeline performance and cost optimization.

• Unlike industry agnostic monitoring agents, Tracer structures DevOps data for scientific pipelines, providing clear visibility into pipeline stages and execution runs. In environments like AWS Batch, where processes and containers are loosely connected, users struggle to understand which processes belong to which pipeline run, and frequently lose logs from failed containers, making debugging difficult.

• Tracer solves this by intelligently organizing and labeling pipelines, execution runs, and steps. Because it runs directly on Linux, it requires no code changes and supports any programming language, unlike point solutions that work only with one framework. This makes integration effortless even across multi-workload IT environments, including AlphaFold, Slurm, Airflow, Nextflow and also local Bash scripts.

• Architected for regulated industries, it ensures enterprise-grade security, with data never leaving your infrastructure, which is not the case with solutions such as DataDog.

Key Features

New metrics that help you speed up your pipelines and maximize your budget:

• Time and cost per dataset processed
• Execution duration and bottleneck identification for each pipeline step
• Cost attribution across pipelines, teams, and environments (dev, CI/CD, prod) Overall, making sense of scientific toolchains with poor/no observability.

Get Started

1. Access the Sandbox

The easiest way to get started with Tracer is via our browser-based sandbox:
👉 https://sandbox.tracer.cloud/

→ Click “Get started” to launch a guided onboarding experience tailored to your preferred tech stack — no AWS credentials or setup required.

2. Install Tracer With One Line of Code

Copy the pre-filled curl command (also shown in the Sandbox) and run it in your terminal:

curl -sSL https://install.tracer.cloud | sh

→ Some environments (e.g. Docker) may require 1-2 additional steps.

3. Initialize a Pipeline

Follow the sandbox instructions to launch your own pipeline or run a script to launch a demo bioinformatics pipeline (Nextflow, WDL, and more) from our nextflow-test-pipelines-bioinformatics repository.

4. Monitor Your Pipeline With Our Grafana Dashboard

Access the Tracer monitoring dashboard on Grafana to watch your pipeline in action, including:

• Real-time execution metrics
• Pipeline stages
• Resource usage across runs

→ The sandbox will guide you through creating your personal account and navigating the Grafana interface.

Mission

"The goal of Tracer's Rust agent is to equip scientists and engineers with DevOps intelligence to efficiently harness massive computational power for humanity's most critical challenges."