Skip to content

epam/cloud-pipeline

Folders and files

NameName
Last commit message
Last commit date

Latest commit

e6176a5 · Jan 24, 2025
Nov 8, 2023
Nov 12, 2024
Oct 21, 2024
Nov 20, 2024
Oct 21, 2024
Dec 25, 2023
Sep 7, 2022
Oct 29, 2024
Jun 15, 2023
Apr 16, 2024
Nov 12, 2024
Feb 15, 2024
Nov 7, 2024
Nov 7, 2024
Nov 5, 2024
Mar 18, 2023
Apr 18, 2023
Mar 22, 2019
Jul 18, 2023
Dec 21, 2021
Aug 20, 2024
Sep 18, 2024
Nov 20, 2024
Jan 24, 2025
Nov 11, 2024
Oct 25, 2024
Oct 21, 2024
Nov 19, 2024
Oct 14, 2024
Oct 21, 2024
Nov 20, 2020
Mar 5, 2021
Mar 22, 2019
Nov 18, 2020
Jul 23, 2024
Dec 21, 2021
Mar 22, 2019
Mar 22, 2019
Aug 26, 2024

Repository files navigation

Cloud Pipeline

Codacy Badge Build status

Cloud Pipeline solution wraps AWS, GCP and Azure compute and storage resources into a single service. Providing an easy and scalable approach to accomplish a wide range of scientific tasks.

  • Data processing: create data processing pipelines and run them in the Cloud in the automated way. Each pipeline represents a workflow script with versioned source code, documentation, and configuration. You can create such scripts in the Cloud Pipeline environment or upload them from the local machine.
  • Data storage management: create your data storage, download or upload data or edit files right in the Cloud Pipeline user interface. File version control is supported.
  • Tools management: create and deploy your own calculation environment using Docker's container concept. Almost every pipeline requires a specific package of software to run it, which is defined in a docker image. So when you start a pipeline, Cloud Pipeline starts a new cloud instance (nodes) and runs a docker image at it.
  • Scientific computing GUI applications: launch and run GUI-based applications using self-service Web interface. It is possible to choose cloud instance configuration, or even use a cluster. Applications are launched as Docker containers exposing Web endpoints or a remote desktop connection (noVNC, NoMachine).

Cloud Pipeline provides a Web-based GUI and also supports CLI, which exposes most of the GUI features.

CP_General

Cloud Pipeline supports Amazon Web Services , Google Cloud Platform and Microsoft Azure Cloud providers to run computing and store data.

Documentation

Detailed documentation on the Cloud Pipeline platform is available via:

Prebuilt binaries

Cloud Pipeline prebuilt binaries are available from the GitHub Releases page