scaleph

The Scaleph project features data integration, develop, job schedule and orchestration and aims to run on cloud environment.

It trys to provide one-stop data platform for simplifying development of data application. Scaleph hopes to help peoples to aggregate and analyze data, free data internal worth and make profit from them.

Scaleph is driven by personal interest and evolves actively through faithful developer, flowerfine is open and appreciates any helps.

Features

• Provide Flink ecosystem to every data developer easily.
• Predefined expert Flink decisions and excellent step wizard.
• Job management.
  • Step job wizard
  • Multiple versions
  • Full job detail web
  • Configuration, dependency and resource

data integration

• Web-ui click-and-drag data integration ways backended by out-of-the-box connectors.

data develop

• Multiple deployment modes and different resource providers.
  • Standalone。Session
  • YARN。Application, Per-Job, Session
  • Kubernetes。Session

Quick Start

Whenever people want to explore Scaleph system, they want a running Scaleph application, then people can interact with Scaleph through Scaleph Admin.

Luckily, deploy Scaleph locally just takes three steps.

• Make sure Docker installed on your machine.
• Clone the repository
• Use Docker Compose and Scaleph Docker image quickly install and run Scaleph.

git clone https://github.com/flowerfine/scaleph.git
cd scaleph/tools/docker/deploy/scaleph
docker-compose up

Once all containers have started, the UI is ready to go at http://localhost, user can login by sys_admin/123456

Documentation

please refer wiki

Build and Deployment

• develop. This doc describes how to set up local development environment of Scaleph project.

• checkstyle. Scaleph project requires clean and robust code, which can help Scaleph go further and develop better.

• build. This doc describes how to compile scaleph-api or scaleph-ui from source and build corresponding docker image. For more information about build from source, refer workflows and actions.

  • compile. Scaleph adopts maven as its build system, .

    • local。People has to install jdk, maven, node, use mvn clean package or npm command to compile project.
    • docker。Recommend way. Just need git and docker, after clone the source code, use docker as the build tools to compile project, which is slower than local.

  • docker. As more application runs in container on cloud then bare metal machine, Scaleph provides own image.

    • docker-image-build。Choose different docker compose yaml file, people can build scalph-api, scaleph-ui or both.
    • docker-image-build-zh。For Chinese user.

• deploy. For different deployment purpose such as develop, test or production, Scaleph make the best effort for people deploy project on local, docker and kubernetes.

  • docker. require fluent network.
  • kubernetes. work in process.

RoadMap

features

1. data ingress and egress.
   1. Flink way data integration . Scaleph reduces complexity of integration through seatunnel, flink-cdc-connectors and other flink connectors.
   2. Friendly web-ui to newbies .
2. data develop
   1. udf + sql.
   2. support multi-layer data warehouse development.
3. job schedule and orchestrate

architectures

1. cloud native
   1. container and kubernetes development and runtime environment.
      1. flink operator
      2. seatunnel operator
      3. scaleph operator
   2. java 17, quarkus.
2. plugins. https://dubbo.apache.org/zh/docsv2.7/dev/principals/

Code of Conduct

This project adheres to the Contributor Covenant code of conduct

Contributing

For contributions, please refer CONTRIBUTING

Thanks for all people who already contributed to Scaleph!

Contact

• Bugs and Features: Issues

• Chinese user:

  

Sponsor

Thanks to JetBrains for supporting us free open source licenses.

License

Scaleph is licenced under the Apache License Version 2.0, link is here.