Skip to content

Latest commit

 

History

History
370 lines (275 loc) · 16.8 KB

README.md

File metadata and controls

370 lines (275 loc) · 16.8 KB
Raw

𐬺 Tinyo

Just orchestrate.

Why • Quickstart • Discord • 𝕏

TIO - A minimal Linux® orchestrator.

1/8 components:

Inspired by Tenstorrent, K8s, tinygrad.

A GitHub ⭐️ would support the open-source ecosystem of TIO in a meaningful way.

Benchmark

Measurements TIO K8s Podman
Container Startup Time (alpine:x.y.z) [s] WIP WIP WIP
Cluster RAM Size [mb] ~288 ~5,476 -
Cluster Disk Size [mb] ~144 ~2,750 -
System Binary Size (total) [mb] ~50 ~2,770 WIP
It's tiny, but it's fast.

System:

Last update: WIP

TIO: (HA, 3 Mesh Nodes)

K8s: (HA, 3 Master, 2 Worker Nodes)

Podman: WIP

☁️ Overview

This is the alpha version of TIO.

Status: We're working on the first MVP (1 node localhost cluster for Linux® & MacOS®).

Challenge

If you can break it, you truly understand it.

If you're able to delete lines and/or break the runtime (tinyort) in any meaningful way, you will be listed in 1337 H4x0r. Please be kind, this is an early alpha. pure-linux-discord

Features

Some of the features of TIO are inspired from k8s/complexities.md.

Simplicity

Cluster

In a nutshell TIO is just an orchestration of runtimes.
  • Efficiency: Lowest possible resource requirements
    • RAM (+ Bandwidth)
    • IOPS
    • Disk
    • ➡️ Significantly smaller than K8s for similar tasks (less cloud/operation costs)
  • Scaling: Infinite (Network mesh)
  • Speed: Fast (Rust & low-level libs, achieving lowest container startup times, WIP)
  • Decentralized mesh
    • No master/worker topology entities.
  • Upgrades: Hustle free (simple system -> better LTS compatibility)
  • Recursive: Cluster in cluster (tint) per default (backup/restore whole clusters with ease using tinyoctl)
  • Dynamicity: E.g. etcd.io for K8s has important limitations we want to address with tinyokv.

Runtime

Optimized Intra-Node Containers

⚡️ Direct Network
Measurements TIO K8s Podman
Container to Container Network Throughput [mb] WIP WIP WIP

TIO introduces an innovative feature for optimizing container communication on the same node, reducing overhead and improving performance. Unlike traditional networking approaches that rely on full TCP/IP stacks or ingress mechanisms, this runtime leverages direct communication methods tailored for Linux architectures. Standard mechanisms will be added in the future to keep more advanced features optional.

Key Features
  • Unix Domain Sockets (UDS): Enables fast and efficient inter-container communication without the overhead of TCP/IP. Each container communicates via UDS managed by the runtime.
  • Shared Memory (shm): Facilitates low-latency data exchange by sharing memory regions between containers for intensive data transfers.
  • Kernel Bypass with eBPF: Utilizes eBPF for advanced packet routing and filtering, optimizing node-level communication pipelines.
  • Direct Process Piping: Supports lightweight communication using POSIX pipes for simpler IPC requirements.
Benefits
  • Performance Boost: Drastically reduces network latency and overhead compared to standard container networking.
  • Secure Isolation: Maintains strict namespace isolation and access control to ensure container security.
  • Customizability: Designed for flexible implementation to fit diverse container workloads.
➕ Instant Duplication
Measurements TIO K8s Podman
1000 alpine:x.y.z Container Parallel Start [s] WIP WIP WIP

This feature introduces an approach to container instantiation based on existing ones in milliseconds on a single node. Instead of traditional container creation workflows, which rely on extracting and initializing container images from scratch, this method enables low-level duplication of existing containers already running on the node. This drastically reduces the time required to spin up identical containers, providing a significant performance boost for scenarios requiring rapid scaling or cloning.

Key Features
  • Low-Level Duplication: Leverages Linux® kernel features to clone running containers directly, bypassing traditional workflows for image extraction and initialization, enabling seamless replication.
  • Blazing Fast Startup Times: Launches new containers in milliseconds by reusing filesystem layers, memory, and resources from existing containers on the same node.
  • Dynamic Resource Optimization: Detects and reuses running containers on the node, minimizing redundant resource allocation and improving performance.
  • Advanced Filesystem Snapshotting: Uses cutting-edge Linux® technologies like overlayfs and copy-on-write (CoW) layers to efficiently duplicate container filesystems without full re-extraction.
  • Secure Namespace Cloning: Maintains container isolation while replicating Linux® namespaces (PID, IPC, UTS, etc.), ensuring secure and consistent environments for each instance.
  • Memory Optimization with KSM: Employs Kernel Same-page Merging (KSM) to share common memory pages between duplicated containers, reducing memory overhead without sacrificing performance.
  • Rust-Powered Efficiency: Built with Rust for high-performance, low-level operations, ensuring safe and reliable container duplication with minimal system impact.
Benefits
  • Rapid Scaling: Scale up workloads almost instantly by duplicating existing containers on-demand.
  • Significant Speedup: Duplication is faster than traditional container creation methods, even those optimized by tools like Kubernetes.

Please give the community feedback about your opinion regarding the development of TIO (Issues/Commits/Discussions).

🏁 Quickstart

Just execute & you're ready to orchestrate on any platform (coming soon). This automatically installs tinyoctl & initializes a local cluster on the host system. Be ready in seconds:

sh curl tinyo.io/get | bash

Deploy a VSCode Server with a user-selected Docker Hub image in just one click and a magic password using tinyocloud (WIP).

Running TIO requires root due to the use of Linux® namespaces, pivot_root, mounting filesystems, modifying network interfaces and so on. Ensure you have the necessary permissions and understand the security implications before using TIO Alpha.

⚠️ The current alpha version of TIO should only be used in a controlled environment, such as a virtual machine (or container, but currently not supported) to prevent unintended side effects on your host system.

Usage

We try to achieve more with less.

TIO uses minimal Statefiles (yml) which are managed via tinyoctl cli.

First setup the first cluster node via tinyoctl cli (alias tio) by providing a network name, username & password of your initial admin user:

tio ++ -net [network_name] -u [admin_username] -p [admin_password]

To add more nodes just execute the command above on a different host system with the following tinyoctl flag in addition:

-ip [IP of any other existing node]

To isolate containers just create additional clusters. A container has network access to any other container in one cluster.

Here is an example to create 1 container with 1 port & 2 mounts:

tio + alpine-1.yml
# Source: https://github.com/pure-linux/tinyoctl/blob/release/alpha/0.0.1/examples/basic/alpine-1.yml
container: "alpine:latest"
port: 8080
mounts:
  - source: "/etc/hosts"
    target: "/mnt/hosts"
  - source: "/tmp/local-config"
    target: "/config"

Architecture

Components

The system has the following 4 main components. tinyoctl interacts with tinyolet to start tinyort. tinyokv is the data store for the TIO cluster node(s). Don't worry about master/worker nodes and so on. The system operates similarly to a mesh topology:

  • tinyoctl: Contains cli and packages intended for use by client programs.
  • tinyolet: Controller for each cluster node. Think of it as biolerplate code which is orchestrating the runtime(s).
  • tinyokv: A high performance distributed Key-Value store for tinyolet (WIP).
  • tinyort: Runtime

E.g. the tinyort/src/utils/core.rs currently has the size of about 350 code lines. The MVP should remain below 2-3k lines for single localhost container with storage & networking (+ Docker Hub download, ..).

To improve TIO and to make it completely accessible to the entire developer community, we introduced the following 3 additional components:

  • tinyoapt: Package Tool to simplify the setup of various applications within a cluster.
  • tinyodash: TIO Pro Dashboard for local development & cloud operations.
  • tinyoctl-k8s-converter: Converts K8s yaml Statefile to TIO Statefile and helm to TIO apt (Vision).

1 security related component:

Vision

Developers want simple, but highly effective tools that reduce (local) infrastructure complexity.

Roadmap

Phase 1: Bicycle

  • Minimal prototype that addresses localhost development with containers
  • MVP version
    • Base cli
    • WIP
  • WIP

Phase 2: Car

  • w clusters in x clusters with y nodes containing z containers
  • WIP

Dashboard Pro

Concept

Visualisation: Full scale TIO cluster with multiple clusters, containers, storage, ingress, pipelines, scales, WIP Source: k8svisual/vpk

PureLinux DAO

➡️ There is a separate repository for the DAO.

Important IP contributors

The people that are listed below made important direct and/or indirect contributions to the vision of TIO. It is very important for us to document any credit regarding our very kind contributors of various kinds and to give them their individual fair stake.

  • [GitHub username] [IP Summary]

Financial Sponsors

We are very grateful for every GitHub/.. Sponsor:

  • [GitHub username]

1337 H4x0r

Experts who successfully deleted lines and/or meaningfully broke the runtime include:

  • Maybe you? pure-linux-discord

PureLinux.org | Delivering to the open-source community what matters most.

Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.