Reliable Nextflow Pipelines with Tracer Observability

Try tracer.cloud instantly in the sandbox environment.

---

Tracer is a system-level observability platform purpose-built for scientific computing. With a one-line Linux client install and instant dashboards, Tracer gives you deep insights into pipeline performance and cost optimization.

---

Why Tracer?

• Reliable Nextflow pipelines: Fully-tested examples that run seamlessly across all environments
• Automated CI/CD: Regularly validated pipelines for consistent functionality
• Easy Installation: Automated scripts for quick setup
• Fast Community Support: Open an issue and get a response within 24 hours
• GitHub Codespaces Compatible: Run examples directly in Codespaces

Quickstart (Sandbox)

Get started instantly:

1. Visit the Tracer Sandbox
2. Follow the onboarding instructions to launch your first monitored pipeline

---

Getting Started

1. Install Tracer

Install Tracer on your system (one-time setup):

Note: Root privileges required

curl -sSL https://install.tracer.cloud | sh && source ~/.bashrc && source ~/.zshrc

2. Initialize the Tracer Client

Start the Tracer client to initialize a pipeline and enable monitoring:

Note: Root privileges required

tracer init

3. Run a Pipeline

You can use your own pipelines or try our ready-to-run examples. For the smoothest onboarding, follow the sandbox instructions or pick a pipeline for your OS below:

• Codespaces
• Linux x86_64
  • Ubuntu
  • Amazon Linux
• Linux aarch64
  • Ubuntu
  • Amazon Linux
• macOS
  • Apple Silicon / arm64
  • Apple Intel / x86
• AWS Batch

We provide a variety of examples for different compute environments, written in Bash, Python, Nextflow, WDL, and CWL.

Recommended: Start with a simple Nextflow fastquorum pipeline.

---

Monitor Your Pipeline

Track your pipeline's progress in real time via the Tracer dashboard. Access it through the 'Open Grafana Dashboard' button during onboarding.

The dashboard provides:

• Execution metrics
• Pipeline stages
• Status updates

---

What Makes Tracer Different?

Tracer is a cutting-edge observability platform designed for scientific computing. Unlike general-purpose monitoring tools, Tracer is purpose-built for scientific pipelines, offering:

• Intelligent organization and labeling of pipelines, runs, and steps
• Zero code changes: runs directly on Linux
• Support for any programming language
• Seamless integration with diverse environments (AlphaFold, Slurm, Airflow, Nextflow, Bash, and more)
• Enterprise-grade security: your data never leaves your infrastructure

This is especially valuable for environments like AWS Batch, where tracking processes across containers is challenging and logs can be lost.

---

Key Features

• Time and cost per dataset processed
• Step-level execution duration and bottleneck identification
• Cost attribution across pipelines, teams, and environments (dev, CI/CD, prod)

These insights help you optimize complex scientific toolchains that traditionally lack proper observability.

---

Mission

"The goal of Tracer's Rust client is to equip scientists and engineers with DevOps intelligence to efficiently harness massive computational power for humanity's most critical challenges."

---

For more details, see the tracer client and our website.