Skip to content

Commit

Permalink
Merge pull request #130 from jjjermiah/development
Browse files Browse the repository at this point in the history
feat: add functionality to prepare filepaths from dicomtags
  • Loading branch information
jjjermiah authored Apr 1, 2024
2 parents c41b230 + dd53773 commit c2e90d1
Show file tree
Hide file tree
Showing 10 changed files with 128 additions and 29 deletions.
33 changes: 13 additions & 20 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,17 @@ on:
# push to any branch *
branches: [ main ]
pull_request:
branches: [ main , development]
branches: [ main , development ]

jobs:
Unit-Tests:
runs-on: ${{ matrix.os }}
timeout-minutes: 15 # Consider increasing timeout

strategy:
matrix:
os: [ubuntu-latest, macos-latest, macos-14]
python-version: ["3.12", "3.11", "3.10"]
# include:
# - os: ubuntu-latest
# python-version: "3.9"

steps:
- uses: actions/checkout@v3
Expand Down Expand Up @@ -123,12 +122,6 @@ jobs:
with:
fetch-depth: 0

# This action uses Python Semantic Release v8
# What this action does:
# - Determines the next version number based on the commit history
# - Creates a new tag with the new version number
# - Pushes the new tag to GitHub
# - Creates a GitHub release with the new version number
- name: Python Semantic Release
id: release
uses: python-semantic-release/python-semantic-release@master
Expand All @@ -140,8 +133,10 @@ jobs:
if: needs.Continuous-Deployment.outputs.released == 'true'
runs-on: ubuntu-latest
steps:
- name: Checkout code
- name: Checkout the code with tag ${{ needs.Continuous-Deployment.outputs.tag }}
uses: actions/checkout@v3
with:
ref: ${{ needs.Continuous-Deployment.outputs.tag }}

- name: Set up Python 3.12
uses: actions/setup-python@v4
Expand Down Expand Up @@ -170,24 +165,22 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
ref: ${{ needs.Continuous-Deployment.outputs.tag }}

- name: Set up QEMU
if: steps.release.outputs.released == 'true'
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
if: steps.release.outputs.released == 'true'
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
if: steps.release.outputs.released == 'true'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Login to the GitHub Container Registry
if: steps.release.outputs.released == 'true'
uses: docker/login-action@v3
with:
registry: ghcr.io
Expand Down Expand Up @@ -234,7 +227,7 @@ jobs:

- name: Install using PyPi
run: |
pip install nbiatoolkit;
pip install nbiatoolkit==${{ needs.Continuous-Deployment.outputs.version }}
NBIAToolkit
Test-Docker-Image:
Expand All @@ -243,12 +236,12 @@ jobs:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
docker_tag: ["latest", "${{ needs.Continuous-Deployment.outputs.tag }}"]

steps:
- uses: actions/checkout@v3
- name: Setup Docker to pull images
uses: docker/setup-buildx-action@v3

- name: Install using Docker
run: |
docker pull ${{ secrets.DOCKERHUB_USERNAME }}/nbiatoolkit:${{ matrix.docker_tag }};
docker run --rm ${{ secrets.DOCKERHUB_USERNAME }}/nbiatoolkit:${{ matrix.docker_tag }} NBIAToolkit
docker pull ${{ secrets.DOCKERHUB_USERNAME }}/nbiatoolkit:${{ needs.Continuous-Deployment.outputs.tag }};
docker run --rm ${{ secrets.DOCKERHUB_USERNAME }}/nbiatoolkit:${{ needs.Continuous-Deployment.outputs.tag }} NBIAToolkit
27 changes: 27 additions & 0 deletions docs/project_info/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,29 @@



## v1.2.0 (2024-04-01)

### Build

* build: Add development branch to CI/CD workflow ([`6ff1e96`](https://github.com/jjjermiah/nbia-toolkit/commit/6ff1e962c981f3a481728b2b0ba03c4d3d9edcc7))

* build: Add timeout for Unit-Tests job and checkout code with tag in Continuous-Deployment job ([`2f85826`](https://github.com/jjjermiah/nbia-toolkit/commit/2f858265dd467267a0079919315280e62c34b173))

### Feature

* feat: release on development ([`b813e2a`](https://github.com/jjjermiah/nbia-toolkit/commit/b813e2a3e82d281f3cdfcb88415100ca451b43a2))

* feat: release on development ([`71e68e0`](https://github.com/jjjermiah/nbia-toolkit/commit/71e68e02ce047331fe1adc7f5a658f9899c8d356))

### Fix

* fix: testing gha ([`272a9f5`](https://github.com/jjjermiah/nbia-toolkit/commit/272a9f52be5f6e1a5f2474c5cc433000a17fa4b6))

### Unknown

* Merge remote-tracking branch 'origin' into development ([`2ed6d37`](https://github.com/jjjermiah/nbia-toolkit/commit/2ed6d37f9d776992bb9bbe23239ce639083aae53))


## v1.1.0 (2024-04-01)

### Build
Expand All @@ -14,6 +37,8 @@

### Chore

* chore(sem-ver): 1.1.0 ([`c41b230`](https://github.com/jjjermiah/nbia-toolkit/commit/c41b2304acd2fc0360ac26ef4736bf10e774871b))

* chore: Update README: 1.0.1 ([`1b7508f`](https://github.com/jjjermiah/nbia-toolkit/commit/1b7508f515ce2820c5b232810fb26660448e66a4))

### Documentation
Expand All @@ -34,6 +59,8 @@

### Fix

* fix: Fix string formatting in version function ([`dc4bbd7`](https://github.com/jjjermiah/nbia-toolkit/commit/dc4bbd749a2b281e81faf7a99a2903bef278beca))

* fix: python 3.9 only on ubuntu ([`1adb7b8`](https://github.com/jjjermiah/nbia-toolkit/commit/1adb7b8c66a6d73fb27374b557eb3b76d9d0c01c))

* fix: Update GitHub Actions workflow to include Ubuntu 3.9 only ([`8d376d3`](https://github.com/jjjermiah/nbia-toolkit/commit/8d376d35f8ef52386330dd8907f35a6ad4a15f4f))
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "nbiatoolkit"
version = "1.1.0"
version = "1.2.0"
description = "A python package to query the National Biomedical Imaging Archive (NBIA) database."
authors = ["Jermiah Joseph"]
license = "MIT"
Expand Down Expand Up @@ -64,7 +64,7 @@ changelog_file = "docs/project_info/CHANGELOG.md"
exclude_commit_types = ["docs", "style", "refactor", "test", "chore"]

[tool.semantic_release.branches.main]
match = "(main|master)"
match = "(main|master|development)"


[tool.semantic_release.commit_parser_options]
Expand Down
1 change: 1 addition & 0 deletions src/nbiatoolkit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from .auth import OAuth2
from .logger.logger import setup_logger
from .utils.nbia_endpoints import NBIA_ENDPOINTS
from .dicomtags import *

# define the __all__ variable
__all__ = [
Expand Down
3 changes: 2 additions & 1 deletion src/nbiatoolkit/dicomsort/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,13 @@

# ]

from .dicomsort import DICOMSorter
from .dicomsort import DICOMSorter, generateFilePathFromDICOMAttributes
from .helper_functions import parseDICOMKeysFromFormat, sanitizeFileName, _truncateUID

__all__ = [
"parseDICOMKeysFromFormat",
"sanitizeFileName",
"_truncateUID",
"DICOMSorter",
"generateFilePathFromDICOMAttributes",
]
7 changes: 6 additions & 1 deletion src/nbiatoolkit/dicomsort/dicomsort.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import shutil
from .helper_functions import parseDICOMKeysFromFormat, sanitizeFileName, _truncateUID
from concurrent.futures import ThreadPoolExecutor, as_completed
from typing import Union


def get_dicom_files(sourceDir) -> list[str]:
Expand All @@ -39,7 +40,7 @@ def read_in_dicom_file(filePath: str) -> pydicom.FileDataset:


def generateFilePathFromDICOMAttributes(
dataset: pydicom.FileDataset,
dataset: pydicom.Dataset,
targetPattern: str,
truncateUID: bool,
sanitizeFilename: bool,
Expand All @@ -57,6 +58,10 @@ def generateFilePathFromDICOMAttributes(
# Retrieve the attribute value if it exists or default to a placeholder string
value = str(getattr(dataset, key, "Unknown" + key))

# if value is exactly "UnknownInstanceNumber", replace it with "1"
if value == "UnknownInstanceNumber":
value = "1"

value = (
_truncateUID(uid=value, lastDigits=5)
if key.endswith("UID") and truncateUID
Expand Down
2 changes: 2 additions & 0 deletions src/nbiatoolkit/dicomtags/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
subsetSeriesTags,
getReferencedFrameOfReferenceSequence,
getReferencedSeriesUIDS,
extract_ROI_info,
)

__all__ = [
Expand All @@ -21,4 +22,5 @@
"subsetSeriesTags",
"getReferencedFrameOfReferenceSequence",
"getReferencedSeriesUIDS",
"extract_ROI_info",
]
30 changes: 28 additions & 2 deletions src/nbiatoolkit/dicomtags/tags.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from math import log
from pydicom.datadict import dictionary_VR
from pydicom.datadict import tag_for_keyword
import pydicom
from pydicom.datadict import dictionary_VR, tag_for_keyword
import pandas as pd
from typing import List

Expand Down Expand Up @@ -342,6 +342,32 @@ def extract_ROI_info(StructureSetROISequence) -> dict[str, dict[str, str]]:
return ROISet


def generateFileDatasetFromTags(tags_df: pd.DataFrame) -> pydicom.Dataset:
"""
Generate a pydicom Dataset object from a DataFrame of DICOM tags.
Args:
tags_df (pd.DataFrame): DataFrame containing DICOM tags.
Returns:
pydicom.Dataset: A pydicom Dataset object containing the DICOM tags.
"""

# Create a new FileDataset
ds = pydicom.Dataset()

for _, row in tags_df.iterrows():
tag = convert_element_to_int(row["element"])
value = row["data"]
if tag == -1:
continue
VR = element_VR_lookup(row["element"])[1]

ds.add_new(tag=tag, VR=VR, value=value)

return ds


# def getRTSTRUCT_ROI_info(seriesUID: str) -> dict[str, dict[str, str]]:
# """
# Given a SeriesInstanceUID of an RTSTRUCT, retrieves the ROI information.
Expand Down
48 changes: 46 additions & 2 deletions src/nbiatoolkit/nbia.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
import re
import zipfile
from tempfile import TemporaryDirectory
from .dicomsort import DICOMSorter

from pydicom import Dataset, FileDataset
from .dicomsort import DICOMSorter, generateFilePathFromDICOMAttributes

import multiprocessing
from .auth import OAuth2
Expand All @@ -26,6 +28,7 @@
getReferencedSeriesUIDS,
extract_ROI_info,
getSequenceElement,
generateFileDatasetFromTags,
)

import pandas as pd
Expand All @@ -38,7 +41,7 @@
from datetime import datetime

# set __version__ variable
__version__ = "1.1.0"
__version__ = "1.2.0"


def downloadSingleSeries(
Expand Down Expand Up @@ -637,6 +640,47 @@ def getRefSeriesUIDs(

return getReferencedSeriesUIDS(series_tags_df=tags_df)

def generateFilePathFromDICOMTags(
self,
SeriesInstanceUID: str,
filePattern: str = "%PatientName/%Modality-%SeriesNumber-%SeriesInstanceUID/%InstanceNumber.dcm",
) -> str:
"""
Generates a file path from DICOM tags.
Args:
SeriesInstanceUID (str): The Series Instance UID of the DICOM series.
filePattern (str, optional): The file pattern to use for generating the file path. Defaults to "%PatientName/%Modality-%SeriesNumber-%SeriesInstanceUID/%InstanceNumber.dcm".
Returns:
str: The generated file path.
Note:
This only considers the first instance of the series.
Meant to be used to determine the dirname of the series files.
"""
self.logger.debug("Getting DICOM tags for series %s", SeriesInstanceUID)
tags_df = self.getDICOMTags(
SeriesInstanceUID=SeriesInstanceUID,
return_type=ReturnType.DATAFRAME,
)

if type(tags_df) != pd.DataFrame:
raise ValueError("DICOM Tags not df or not found in the response.")

self.logger.debug("Generating file path from DICOM tags")
ds: Dataset = generateFileDatasetFromTags(tags_df=tags_df)
filePath: str = generateFilePathFromDICOMAttributes(
dataset=ds,
targetPattern=filePattern,
truncateUID=True,
sanitizeFilename=True,
)
self.logger.debug(
"Generated file path: %s for series %s", filePath, SeriesInstanceUID
)
return filePath

def downloadSeries(
self,
SeriesInstanceUID: Union[str, list],
Expand Down
2 changes: 1 addition & 1 deletion src/nbiatoolkit/nbia_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


def version():
f = """
f = r"""
_ ______ _______ ______ ____ _ __
/ | / / __ )/ _/ |/_ __/___ ____ / / /__(_) /_
/ |/ / __ |/ // /| | / / / __ \/ __ \/ / //_/ / __/
Expand Down

0 comments on commit c2e90d1

Please sign in to comment.