From 8e0a0be43114b1726d1a2629769d6fdd6d76f7de Mon Sep 17 00:00:00 2001 From: Avi Lumelsky Date: Mon, 7 Oct 2024 12:47:02 +0300 Subject: [PATCH] Update README.md --- README.md | 344 +++++++++++++++--------------------------------------- 1 file changed, 91 insertions(+), 253 deletions(-) diff --git a/README.md b/README.md index 228f9e4..3beea27 100644 --- a/README.md +++ b/README.md @@ -1,293 +1,131 @@ # secimport + [![Upload Python Package](https://github.com/avilum/secimport/actions/workflows/python-publish.yml/badge.svg)](https://github.com/avilum/secimport/actions/workflows/python-publish.yml) ![](https://img.shields.io/badge/Test_Coverage-90%-blue) -A Tailor-Made Sandbox for Your Python Applications.
-secimport is eBPF-based sandbox toolkit for Python, that enforces specific syscalls per module in your code.
-1. It traces your python code -2. It creates a security profile for your application
-3. Use the profile by applying it to running processes, or to run a new python process with supervision. - -secimport for python is like seccomp-bpf for linux but per module in your code. - +## Module-Level Sandboxing for Python Applications +secimport is an eBPF-based security toolkit that enforces syscall restrictions per Python module, providing granular control over your application's security profile. Think of it as seccomp-bpf for Linux, but operating at the Python module level. - - - - Star History Chart - +  +    +    +   Star History Chart +  +## Key Features +- **Module-Level Security**: Define and enforce syscall restrictions per Python module +- **Automated Profiling**: Traces your application to create tailored security profiles +- **Multiple Enforcement Modes**: Log, stop, or kill processes on policy violations +- **Production Ready**: Negligible performance impact thanks to eBPF +- **Supply Chain Protection**: Mitigate risks from vulnerable dependencies - - -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - -- [Background](#background) - - [Technical Blogs](#technical-blogs) - - [The problem](#the-problem) - - [How it works](#how-it-works) - - [Who is it for?](#who-is-it-for) -- [Getting Started](#getting-started) - - [Example Sandboxes](#example-sandboxes) - - [Using Docker](#using-docker) - - [Without Docker](#without-docker) -- [Quickstart: trace, build, run.](#quickstart-trace-build-run) -- [Command Line Usage](#command-line-usage) - - [Stop on violation](#stop-on-violation) - - [Kill on violation](#kill-on-violation) -- [Advanced](#advanced) - - [Python API](#python-api) - - [seccomp-bpf support using nsjail](#seccomp-bpf-support-using-nsjail) -- [Changelog](#changelog) - - [Contributing](#contributing) - - - -# Background -## Talks -[![Secimport Talk on BSIDES](https://img.youtube.com/vi/nRV0ulYMsxU/0.jpg)](https://youtu.be/nRV0ulYMsxU?t=1257) -## Technical Blogs -- secimport + Dtrace -- secimprt + eBPF + PyTorch -- secimport + eBPF + FastAPI - -## The problem -Traditional tools like seccomp or AppArmor enforce syscalls for the entire process.
Something like `allowed_syscalls=["read","openat","bind","write"]`, which is great, but not enough for python's attack surface.
- -## How it works -1. `secimport` is able to trace which syscalls each module in your code uses (by package name).
-2. After tracing, secimport creates a JSON/YAML policy for your code. It compiles it into a high-performance eBPF instrumentation script (smaller then 512 bytes overall), that looks like this. - ``` - modules: - requests: - destructive: true # when true, secimport will kill on vilation instead of logging. - syscall_allowlist: - - fchmod - - getentropy - - getpgrp - - getrlimit - ... - ``` - You can also use JSON instaead of YAML, and even confine builtin python modules.
-3. Finally, you convert this policy into an sandbox (eBPF instrumentation script) to run the python process in production. Running the sandbox will enfore python to obey any given policy. - -## Who is it for? -`secimport` is great for... -- Preventing Code Execution: reduce the risk of supply chain attacks. - - Trace the syscalls flow of your application at the user-space/os/kernel level and per module. - - Run your application while enforcing syscalls per module. - - Upon violation of the policy, it can log, stop, or kill the process. -- Protect yourself from RCE: - - secimport makes 1day attacks less of an issue, because it prevents the code form running. If you are using a vulnerable package and someone exploited it, your policy will not allow this exploit's syscalls and it will be handled as you wish. - - Avoid incidents like log4shell. A logging library requires very few syscalls, and it should never run command using fork, execve or spawn. - - The syscalls that "fastapi", "numpy" or "requests" use are very different. -- Load AI Models from Insecure Sources - - Models from unsafe source (huggingface, torch hub, and pytorch pickled models) can be limited to run only a set of syscalls. RCE like 'import os;os.system(...)' somewhere deep in the code will be catched by secimport. -- Minimal Performance Impact - - Has negligible performance impact and is production-ready thanks to eBPF. Check out the [Performance](https://github.com/avilum/secimport/wiki/Performance-Benchmarks) benchmarks. -- Trace which syscalls are called by each module in your code. - - secimport uses USDT (Userland Statically Defined Tracing) together with kernel probes in the runtime using eBPF or dtrace instrumentation scripts. - - -# Getting Started -## Example Sandboxes - -The [Sandbox Examples](https://github.com/avilum/secimport/wiki/Sandbox-Examples) page contains basic and advanced real-world examples. -## Using Docker -The quickest way to evaluate `secimport` is to use our [Docker container](docker/README.md), which includes `bpftrace` (`ebpf`) and other plug-and-play examples. -For quicker evaluation, we recommend using the Secimport Docker Image instead of self-installing.
-- Build and run the Docker container with a custom kernel that matches your existing OS kernel version: - ``` - cd docker/ && ./build.sh && ./run.sh - ``` - A temporary container will be created, and you will be logged in as the root user. - -## Without Docker -Tested on Ubuntu, Debian, Rocky (Linux x86/AMD/ARM) and MacOS in (x86/M1). If you run on MacOS you will need to disable SIP for dtrace. - -1. Install python with USDT probes by configuring it with '--dtrace' -2. Install one of the backends: eBPF or DTrace. -3. Install secimport - - Install from pypi - - ``` - python3 -m pip install secimport - ``` - - Install from source - - ``` - git clone https://github.com/avilum/secimport.git && cd secimport - python3 -m pip install poetry && python3 -m poetry install - ``` - -# Quickstart: trace, build, run. -```shell -root@1fa3d6f09989:/workspace# secimport interactive - -Let's create our first tailor-made sandbox with secimport! -- A python shell will be opened -- The behavior will be recorded. - -OK? (y): y - >>> secimport trace - -TRACING: ['/workspace/secimport/profiles/trace.bt', '-c', '/workspace/Python-3.11.8/python', '-o', 'trace.log'] - - Press CTRL+D to stop the trace; - -Python 3.11.8 (default, Mar 19 2023, 08:34:46) [GCC 9.4.0] on linux -Type "help", "copyright", "credits" or "license" for more information. ->>> import this ->>> - TRACING DONE; - >>> secimport build - -SECIMPORT COMPILING... - -CREATED JSON TEMPLATE: policy.json -CREATED YAML TEMPLATE: policy.yaml -compiling template policy.yaml -DTRACE SANDBOX: sandbox.d -BPFTRCE SANDBOX: sandbox.bt -``` +## Quick Start -Now, let's run the sandbox! -```python -- Run the same commands as before, they should run without any problem;. -- Do something new in the shell; e.g: >>> __import__("os").system("ps") - - OK? (y): y - >>> secimport run - RUNNING SANDBOX... ['./sandbox.bt', '--unsafe', ' -c ', '/workspace/Python-3.11.8/python'] -Attaching 5 probes... -REGISTERING SYSCALLS... -STARTED -Python 3.11.8 (default, Mar 19 2023, 08:34:46) [GCC 9.4.0] on linux -Type "help", "copyright", "credits" or "license" for more information. ->>> import this ->>> import os -[SECIMPORT VIOLATION]: called syscall ioctl at depth 0 -[SECIMPORT VIOLATION]: called syscall ioctl at depth 0 -``` +### Using Docker (Recommended) -For more detailed usage instructions, see the [Command-Line Usage](https://github.com/avilum/secimport/wiki/Command-Line-Usage) page. +```bash +git clone https://github.com/avilum/secimport.git +cd secimport/docker +./build.sh && ./run.sh +``` -# Command Line Usage -To sandbox your program using the CLI, start a bpftrace program that logs all the syscalls for all the modules in your application into a file with the secimport trace command. Once you have covered the logic you would like to sandbox, hit CTRL+C or CTRL+D, or wait for the program to finish. Then, build a sandbox from the trace using the secimport build command, and run the sandbox with the secimport run command. +### Manual Installation -```shell -NAME - secimport - is a comprehensive toolkit designed to enable the tracing, construction, and execution of secure Python runtimes. It leverages USDT probes and eBPF/DTrace technologies to enhance the overall security measures. +1. Install Python with USDT probes: + ```bash + # Configure Python with --enable-dtrace + # See detailed instructions in our wiki + ``` -SYNOPSIS - secimport COMMAND +2. Install a supported backend (eBPF or DTrace) + ```bash + # Ubuntu/Debian + apt-get install bpftrace -DESCRIPTION - https://github.com/avilum/secimport/wiki/Command-Line-Usage + # For other platforms, see our Installation wiki + ``` - WORKFLOW: - 1. secimport trace / secimport shell - 2. secimport build - 3. secimport run +3. Install secimport + ```bash + pip install secimport + ``` - QUICKSTART: - $ secimport interactive +## Creating Your First Sandbox - EXAMPLES: - 1. trace: - $ secimport trace - $ secimport trace -h - $ secimport trace_pid 123 - $ secimport trace_pid -h - 2. build: - # secimport build - $ secimport build -h - 3. run: - $ secimport run - $ secimport run --entrypoint my_custom_main.py - $ secimport run --entrypoint my_custom_main.py --stop_on_violation=true - $ secimport run --entrypoint my_custom_main.py --kill_on_violation=true - $ secimport run --sandbox_executable /path/to/my_sandbox.bt --pid 2884 - $ secimport run --sandbox_executable /path/to/my_sandbox.bt --sandbox_logfile my_log.log - $ secimport run -h +```bash +secimport interactive -COMMANDS - COMMAND is one of the following: +# In the Python shell that opens: +>>> secimport trace # Start tracing +>>> import requests # Perform actions you want to profile +>>> # Press CTRL+D to stop tracing - build - Compiles a trace log (trace.log). Creates the sandbox executable (instrumentation script) for each supported backend It uses `create_profile_from_trace ...` and `sandbox_from_profile`. +>>> secimport build # Build sandbox from trace +>>> secimport run # Run with enforcement +``` - compile_sandbox_from_profile - Generates a tailor-made sandbox that will enforce a given yaml profile/policy in runtime. +## Advanced Usage - interactive +### Command Line Options - run - Run a python process inside the sandbox. +```bash +secimport trace # Trace a new Python process +secimport trace_pid # Trace an existing process +secimport build # Build sandbox from trace +secimport run [options] # Run with enforcement +``` - shell - Alternative syntax for secimport "trace". +### Enforcement Modes - trace - Traces a python process using an entrypoint or interactive interpreter. It might require sudo privilleges on some hosts. +```bash +# Stop on violation +secimport run --stop_on_violation=true - trace_pid - Traces a running process by pid. It might require sudo privilleges on some hosts. +# Kill on violation +secimport run --kill_on_violation=true ``` -### Stop on violation -``` -root@1bc0531d91d0:/workspace# secimport run --stop_on_violation=true - >>> secimport run -[WARNING]: This sandbox will send SIGSTOP to the program upon violation. - RUNNING SANDBOX... ['./sandbox.bt', '--unsafe', ' -c ', '/workspace/Python-3.11.8/python', 'STOP'] -Attaching 4 probes... -Python 3.11.8 (default, Apr 28 2023, 11:32:40) [GCC 9.4.0] on linux -Type "help", "copyright", "credits" or "license" for more information. ->>> import os ->>> os.system('ps') -[SECURITY PROFILE VIOLATED]: called syscall 56 at depth 8022 - -^^^ STOPPING PROCESS 85918 DUE TO SYSCALL VIOLATION ^^^ - PROCESS 85918 STOPPED. -``` - -### Kill on violation -``` -root@ee4bc99bb011:/workspace# secimport run --kill_on_violation - >>> secimport run -[WARNING]: This sandbox will send SIGKILL to the program upon violation. - RUNNING SANDBOX... ['./sandbox.bt', '--unsafe', ' -c ', '/workspace/Python-3.11.8/python', 'KILL'] -import os -oAttaching 4 probes... -sPython 3.11.8 (default, Apr 28 2023, 11:32:40) [GCC 9.4.0] on linux -Type "help", "copyright", "credits" or "license" for more information. ->>> import os ->>> os.system('ps') -[SECURITY PROFILE VIOLATED]: called syscall 56 at depth 8022 - -^^^ KILLING PROCESS 86466 DUE TO SYSCALL VIOLATION ^^^ - KILLED. - SANDBOX EXITED; -``` +### Python API +```python +import secimport -# Advanced +# Replace standard import with secure import +requests = secimport.secure_import('requests', allowed_syscalls=['open', 'read', ...]) +``` -### Python API -See the [Python Imports](examples/python_imports/) example for more details.
-You can use secimport directly from python, instead of the CLI.
you can replace `import` with `secimport.secure_import` for selected modules, and have them supervised in real time.
+## seccomp-bpf support using nsjail -### seccomp-bpf support using nsjail Beside the sandbox that secimport builds,
The `secimport build` command creates an nsjail sandbox with seccomp profile for your traced code.
`nsjail` enables namespace sandboxing with seccomp on linux
`secimport` automatically generates seccomp profiles to use with `nsjail` as executable bash script. It can be used to limit the syscalls of the entire python process, as another layer of defence. -# Changelog -See the [Changelog](https://github.com/avilum/secimport/blob/master/docs/CHANGELOG.md) for development progress and existing features. +## Documentation + +- [Installation Guide](https://github.com/avilum/secimport/wiki/Installation) +- [Command Line Usage](https://github.com/avilum/secimport/wiki/Command-Line-Usage) +- [API Reference](https://github.com/avilum/secimport/wiki/Python-API) +- [Example Sandboxes](https://github.com/avilum/secimport/wiki/Sandbox-Examples) + +## Learn More + +### Technical Resources +- https://www.oligo.security/ +- [Talk: secimport at BSides](https://youtu.be/nRV0ulYMsxU?t=1257) +- Blog Posts: + - [secimport + DTrace](https://infosecwriteups.com/sandboxing-python-modules-in-your-code-1e590d71fc26?source=friends_link&sk=5e9a2fa4d4921af0ec94f175f7ee49f9) + - [secimport + eBPF + PyTorch](https://infosecwriteups.com/securing-pytorch-models-with-ebpf-7f75732b842d?source=friends_link&sk=14d8db403aaf66724a8a69b4dea24e12) + - [secimport + eBPF + FastAPI](https://avi-lumelsky.medium.com/secure-fastapi-with-ebpf-724d4aef8d9e?source=friends_link&sk=b01a6b97ef09003b53cd52c479017b03) + +## Contributing + +We welcome contributions! See our [Contributing Guide](https://github.com/avilum/secimport/blob/master/docs/CONTRIBUTING.md) for details. + +## License -### Contributing -For information on how to contribute to `secimport`, see the [Contributing](https://github.com/avilum/secimport/blob/master/docs/CONTRIBUTING.md) guide. +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.