diff --git a/.github/workflows/gitleaks.yml b/.github/workflows/gitleaks.yml new file mode 100644 index 000000000..4accdeb19 --- /dev/null +++ b/.github/workflows/gitleaks.yml @@ -0,0 +1,57 @@ +name: "Gitleaks" + +on: + pull_request: + +env: + GIT_GITLEAKS_VERSION: 8.18.4 + ACTIONS_STEP_DEBUG: '${{ secrets.ACTIONS_STEP_DEBUG }}' + +jobs: + gitleaks: + name: "Gitleaks" + runs-on: ubuntu-latest + steps: + - uses: hmarr/debug-action@v3.0.0 + + - name: Configure gitleaks binary cache + id: cache + uses: actions/cache@v4 + with: + path: /usr/local/bin/gitleaks + key: gitleaks-${{ env.GIT_GITLEAKS_VERSION }} + + - name: Install tools + if: steps.cache.outputs.cache-hit != 'true' + run: | + wget -O - \ + "https://github.com/zricethezav/gitleaks/releases/download/v${{ env.GIT_GITLEAKS_VERSION }}/gitleaks_${{ env.GIT_GITLEAKS_VERSION }}_linux_x64.tar.gz" \ + | sudo tar xzf - -C /usr/local/bin + + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 + ref: ${{ github.event.pull_request.head.sha || github.event.after }} + + - name: Run gitleaks + run: | + set -euo pipefail ${ACTIONS_STEP_DEBUG:+-x} + + gitleaks \ + detect \ + --source="." \ + --redact \ + -v \ + --exit-code=2 \ + --report-format=sarif \ + --report-path=results.sarif \ + --log-level=debug \ + --log-opts='${{ github.event.pull_request.base.sha || github.event.before }}..${{ github.event.pull_request.head.sha || github.event.after }}' + + - name: Upload test results + if: failure() + uses: actions/upload-artifact@v4 + with: + name: GitLeaks results + path: results.sarif \ No newline at end of file diff --git a/README.md b/README.md index 09083255d..64f06fae5 100644 --- a/README.md +++ b/README.md @@ -61,6 +61,8 @@ Check the **Required parameters** column to see if you need to set any additiona | [spacy](/label_studio_ml/examples/spacy) | NER by [SpaCy](https://spacy.io/) | ✅ | ❌ | ❌ | None | Set [(see documentation)](https://spacy.io/usage/linguistic-features) | | [tesseract](/label_studio_ml/examples/tesseract) | Interactive OCR. [Details](https://github.com/tesseract-ocr/tesseract) | ❌ | ✅ | ❌ | None | Set (characters) | | [watsonX](/label_studio_ml/exampels/watsonx)| LLM inference with [WatsonX](https://www.ibm.com/products/watsonx-ai) and integration with [WatsonX.data](watsonx.data)| ✅ | ✅| ❌ | None| Arbitrary| +| [yolo](/label_studio_ml/examples/yolo) | Object detection with [YOLO](https://docs.ultralytics.com/tasks/) | ✅ | ❌ | ❌ | None | Arbitrary | + # (Advanced usage) Develop your model To start developing your own ML backend, follow the instructions below. diff --git a/label_studio_ml/examples/grounding_dino/README.md b/label_studio_ml/examples/grounding_dino/README.md index 454c3cc30..5be83ec77 100644 --- a/label_studio_ml/examples/grounding_dino/README.md +++ b/label_studio_ml/examples/grounding_dino/README.md @@ -40,7 +40,7 @@ See [here](https://github.com/IDEA-Research/GroundingDINO) for more details abou Example: - `LABEL_STUDIO_HOST=http://123.456.7.8:8080` - - `LABEL_STUDIO_ACCESS_TOKEN=c9djf998eii2948ee9hh835nferkj959923` + - `LABEL_STUDIO_ACCESS_TOKEN=your-api-key` 3. Run `docker compose up` 4. Check the IP of your backend using `docker ps`. You will use this URL when connecting the backend to a Label Studio project. Usually this is `http://localhost:9090`. diff --git a/label_studio_ml/examples/grounding_sam/README.md b/label_studio_ml/examples/grounding_sam/README.md index f03d98a79..5ed4bbb9c 100644 --- a/label_studio_ml/examples/grounding_sam/README.md +++ b/label_studio_ml/examples/grounding_sam/README.md @@ -43,7 +43,7 @@ See [here](https://github.com/IDEA-Research/GroundingDINO) for more details abou Example: - `LABEL_STUDIO_HOST=http://123.456.7.8:8080` - - `LABEL_STUDIO_ACCESS_TOKEN=c9djf998eii2948ee9hh835nferkj959923` + - `LABEL_STUDIO_ACCESS_TOKEN=your-api-key` 3. Run `docker compose up` 4. Check the IP of your backend using `docker ps`. You will use this URL when connecting the backend to a Label Studio project. Usually this is `http://localhost:9090`. diff --git a/label_studio_ml/examples/grounding_sam/docker-compose.yml b/label_studio_ml/examples/grounding_sam/docker-compose.yml index 4b66e6d9c..a1acf99fd 100644 --- a/label_studio_ml/examples/grounding_sam/docker-compose.yml +++ b/label_studio_ml/examples/grounding_sam/docker-compose.yml @@ -3,7 +3,7 @@ version: "3.11" services: grounding_sam: container_name: grounding_sam - image: heartexlabs/label-studio-ml-backend:grndsam-master + image: heartexlabs/label-studio-ml-backend:grounding_sam-master build: context: . args: diff --git a/label_studio_ml/examples/mmdetection-3/mmdetection.py b/label_studio_ml/examples/mmdetection-3/mmdetection.py index 9bf28e82f..622d8bb01 100644 --- a/label_studio_ml/examples/mmdetection-3/mmdetection.py +++ b/label_studio_ml/examples/mmdetection-3/mmdetection.py @@ -100,6 +100,7 @@ def build_labels_from_labeling_config(self, schema): for ls_label, label_attrs in self.labels_attrs.items(): predicted_values = label_attrs.get("predicted_values", "").split(",") for predicted_value in predicted_values: + predicted_value = predicted_value.strip() # remove spaces at the beginning and at the end if predicted_value: # it shouldn't be empty (like '') if predicted_value not in mmdet_labels: print( diff --git a/label_studio_ml/examples/mmdetection-3/requirements-base.txt b/label_studio_ml/examples/mmdetection-3/requirements-base.txt index d68a65cfd..e7661626e 100644 --- a/label_studio_ml/examples/mmdetection-3/requirements-base.txt +++ b/label_studio_ml/examples/mmdetection-3/requirements-base.txt @@ -1,2 +1,2 @@ gunicorn==22.0.0 -label-studio-ml @ git+https://github.com/HumanSignal/label-studio-ml-backend.git +label-studio-ml @ git+https://github.com/HumanSignal/label-studio-ml-backend.git@fix/rnd-117 diff --git a/label_studio_ml/examples/mmdetection-3/test_model.py b/label_studio_ml/examples/mmdetection-3/test_model.py index 329c67d39..33db0b37a 100644 --- a/label_studio_ml/examples/mmdetection-3/test_model.py +++ b/label_studio_ml/examples/mmdetection-3/test_model.py @@ -2,7 +2,7 @@ from mmdetection import MMDetection -from pytest import approx +from label_studio_ml.utils import compare_nested_structures label_config = """ @@ -41,22 +41,6 @@ ] -def compare_nested_structures(a, b, path=""): - """Compare two dicts or list with approx() for float values""" - if isinstance(a, dict) and isinstance(b, dict): - assert a.keys() == b.keys(), f"Keys mismatch at {path}" - for key in a.keys(): - compare_nested_structures(a[key], b[key], path + "." + str(key)) - elif isinstance(a, list) and isinstance(b, list): - assert len(a) == len(b), f"List size mismatch at {path}" - for i, (act_item, exp_item) in enumerate(zip(a, b)): - compare_nested_structures(act_item, exp_item, path + f"[{i}]") - elif isinstance(a, float) and isinstance(b, float): - assert a == approx(b), f"Mismatch at {path}" - else: - assert a == b, f"Mismatch at {path}" - - def test_mmdetection_model_predict(): model = MMDetection(label_config=label_config) predictions = model.predict([task]) diff --git a/label_studio_ml/examples/yolo/.dockerignore b/label_studio_ml/examples/yolo/.dockerignore new file mode 100644 index 000000000..fee972bd7 --- /dev/null +++ b/label_studio_ml/examples/yolo/.dockerignore @@ -0,0 +1,22 @@ +# Exclude everything +** + +# Include Dockerfile and docker-compose for reference (optional, decide based on your use case) +!Dockerfile +!docker-compose.yml + +# Include Python application files +!*.py +!*.yaml +!tests/* +!control_models/* +!models/* + +# Include requirements files +!requirements*.txt + +# Include script +!*.sh + +# Exclude specific requirements if necessary +# requirements-test.txt (Uncomment if you decide to exclude this) diff --git a/label_studio_ml/examples/yolo/Dockerfile b/label_studio_ml/examples/yolo/Dockerfile new file mode 100644 index 000000000..ff417da64 --- /dev/null +++ b/label_studio_ml/examples/yolo/Dockerfile @@ -0,0 +1,62 @@ +FROM pytorch/pytorch:2.1.2-cuda12.1-cudnn8-runtime +ARG DEBIAN_FRONTEND=noninteractive +ARG TEST_ENV + +WORKDIR /app + +RUN conda update conda -y + +RUN --mount=type=cache,target="/var/cache/apt",sharing=locked \ + --mount=type=cache,target="/var/lib/apt/lists",sharing=locked \ + apt-get -y update \ + && apt-get install -y git \ + && apt-get install -y wget \ + && apt-get install -y g++ freeglut3-dev build-essential libx11-dev \ + libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-dev libfreeimage-dev \ + && apt-get -y install ffmpeg libsm6 libxext6 libffi-dev python3-dev python3-pip gcc + +ENV PYTHONUNBUFFERED=1 \ + PYTHONDONTWRITEBYTECODE=1 \ + PIP_CACHE_DIR=/.cache \ + PORT=9090 \ + WORKERS=2 \ + THREADS=4 \ + CUDA_HOME=/usr/local/cuda + +RUN conda install -c "nvidia/label/cuda-12.1.1" cuda -y +ENV CUDA_HOME=/opt/conda \ + TORCH_CUDA_ARCH_LIST="6.0;6.1;7.0;7.5;8.0;8.6+PTX;8.9;9.0" + +# install base requirements +COPY requirements-base.txt . +RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \ + pip install -r requirements-base.txt + +# install model requirements +COPY requirements.txt . +RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \ + pip3 install -r requirements.txt + +# install test requirements if needed +COPY requirements-test.txt . +# build only when TEST_ENV="true" +RUN --mount=type=cache,target=${PIP_CACHE_DIR},sharing=locked \ + if [ "$TEST_ENV" = "true" ]; then \ + pip3 install -r requirements-test.txt; \ + fi + +WORKDIR /app + +COPY . ./ + +WORKDIR /app/models + +# Download the YOLO models +RUN yolo predict model=yolov8m.pt source=/app/tests/car.jpg \ + && yolo predict model=yolov8n.pt source=/app/tests/car.jpg \ + && yolo predict model=yolov8n-cls.pt source=/app/tests/car.jpg \ + && yolo predict model=yolov8n-seg.pt source=/app/tests/car.jpg + +WORKDIR /app + +CMD ["/app/start.sh"] diff --git a/label_studio_ml/examples/yolo/README.md b/label_studio_ml/examples/yolo/README.md new file mode 100644 index 000000000..d618fdf6c --- /dev/null +++ b/label_studio_ml/examples/yolo/README.md @@ -0,0 +1,810 @@ + + +# YOLO ML backend for Label Studio + +The YOLO ML backend for Label Studio is designed to integrate advanced object detection, +segmentation, classification, and video object tracking capabilities directly into Label Studio. + +This integration allows you to leverage powerful YOLOv8 models for various machine learning tasks, +making it easier to annotate large datasets and ensure high-quality predictions. + +
+ + YOLO Integration Video +
+ Check the video tutorial +
+
+
+ +**Supported Features** + +| YOLO Task Name | LS Control Tag | Prediction Supported | LS Import Supported | LS Export Supported | +|---------------------------------------|--------------------------------------|----------------------|---------------------|---------------------| +| Object Detection | `` | ✅ | YOLO, COCO | YOLO, COCO | +| Oriented Bounding Boxes (OBB) | `` | ✅ | YOLO | YOLO | +| Image Instance Segmentation: Polygons | `` | ✅ | COCO | YOLO, COCO | +| Image Semantic Segmentation: Masks | `` | ❌ | Native | Native | +| Image Classification | `` | ✅ | Native | Native | +| Pose Detection | `` | ✅ | Native | Native | +| Video Object Tracking | `` | ✅ | Native | Native | +| Video Temporal Classification | `` | Coming soon | Native | Native | + +* **LS Control Tag**: Label Studio [control tag](https://labelstud.io/tags/) from the labeling configuration. +* **LS Import Supported**: Indicates whether Label Studio supports Import from YOLO format to Label Studio (using the LS converter). +* **LS Export Supported**: Indicates whether Label Studio supports Export from Label Studio to YOLO format (the **Export** button on the Data Manager and using the LS converter). +* **Native**: Native means that only native Label Studio JSON format is supported. + + +## Before you begin + +Before you begin, you need to install the [Label Studio ML backend](https://github.com/HumanSignal/label-studio-ml-backend?tab=readme-ov-file#quickstart). + +This tutorial uses the [YOLO example](https://github.com/HumanSignal/label-studio-ml-backend/tree/master/label_studio_ml/examples/yolo). + +## Quick start + +1. Add `LABEL_STUDIO_URL` and `LABEL_STUDIO_API_KEY` to the `docker-compose.yml` file. These variables should point to your Label Studio instance and its API key, respectively. For more information about finding your Label Studio API key, [see our documentation](https://labelstud.io/guide/user_account#Access-token). + +2. Run docker compose + + ```bash + docker-compose up --build + ``` + +3. Open Label Studio and create a new project with the following labeling config: + + ```xml + + + + + + ``` + +4. Then from the **Model** page in the project settings, [connect the model](https://labelstud.io/guide/ml#Connect-the-model-to-Label-Studio). The default URL is `http://localhost:9090`. + +5. Add images to Label Studio. + +6. Open any task in the Data Manager and see the predictions from the YOLO model. + +## Labeling configurations + +### Supported object & control tags + +**Object tags** + +- `` - [Image to annotate](https://labelstud.io/tags/image) +- `