Fixes for pydicom 3 (#301)

* Import related fixes; remove fix_meta_info, now unnecessary

* Drop support for python<3.10

* Fixes to codes and RLE encoding

* Remove fix for resolved pydicom bug in decode_frame

* Fix frame encoding and decoding, add further transfer syntaxes

* Various fixes

* Enforce repo-review rules (#296)

* Enforce repo-review rule PP302

PP302: Sets a minimum pytest to at least 6

Must have a `minversion=`, and must be at least 6 (first version to
support `pyproject.toml` configuration).

* Enforce repo-review rule PP305

PP305: Specifies xfail_strict

`xfail_strict` should be set. You can manually specify if a check should
be strict when setting each xfail.

* Enforce repo-review rule PP306

PP306: Specifies strict config

`--strict-config` should be in `addopts = [...]`. This forces an error
if a config setting is misspelled.

* Enforce repo-review rule PP307

PP307: Specifies strict markers

`--strict-markers` should be in `addopts = [...]`. This forces all
markers to be specified in config, avoiding misspellings.

* Enforce repo-review rule PP308

PP308: Specifies useful pytest summary

An explicit summary flag like `-ra` should be in `addopts = [...]` (print
summary of all fails/errors).

* Enforce pytest ≥ 7.3.2

This is the first version to support Python 3.12:
https://docs.pytest.org/en/stable/changelog.html#pytest-7-3-2-2023-06-10

* Enforce repo-review rule MY104

MY104: MyPy enables ignore-without-code

Must have `"ignore-without-code"` in `enable_error_code = [...]`. This
will force all skips in your project to include the error code, which
makes them more readable, and avoids skipping something unintended.

* Enforce repo-review rule MY105

MY105: MyPy enables redundant-expr

Must have `"redundant-expr"` in `enable_error_code = [...]`. This helps
catch useless lines of code, like checking the same condition twice.

* Enforce repo-review rule MY106

MY106: MyPy enables truthy-bool

Must have `"truthy-bool"` in `enable_error_code = []`. This catches
mistakes in using a value as truthy if it cannot be falsy.

* Remove pillow-jpls dependency as functionality now comes from pylibjpeg

* Add details of single bit JPEG2000 to the docs

* Better workaround for floating point pixel decoding

* Adjust dependencies

* Deprecate row_stack for vstack

* Bump pydicom version to 3.0.1

* Error message typo

---------

Co-authored-by: Dimitri Papadopoulos Orfanos <[email protected]>

.github/workflows/run_unit_tests.yml

@@ -15,7 +15,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.10", "3.11", "3.12"]
         dependencies: [".", "'.[libjpeg]'"]
 
     steps:

docs/seg.rst

@@ -807,39 +807,38 @@ We recommend that if you do this, you specify ``max_fractional_value=1`` to
 clearly communicate that the segmentation is inherently binary in nature.
 
 Why would you want to make this seemingly rather strange choice? Well,
-``"FRACTIONAL"`` SEGs tend to compress *much* better than ``"BINARY"`` ones
-(see next section). Note however, that this is arguably an misuse of the intent
-of the standard, so *caveat emptor*.
+``"FRACTIONAL"`` SEGs tend to compress better than ``"BINARY"`` ones (see next
+section). Note however, that this is arguably an misuse of the intent of the
+standard, so *caveat emptor*. Also note that while this used to be a more
+serious issue it is less serious now that ``"JPEG2000Lossless"`` compression is
+now supported for ``"BINARY"`` segmentations as of highdicom v0.23.0.
 
 Compression
 -----------
 
 The types of pixel compression available in segmentation images depends on the
-segmentation type. Pixels in a ``"BINARY"`` segmentation image are "bit-packed"
-such that 8 pixels are grouped into 1 byte in the stored array. If a given frame
-contains a number of pixels that is not divisible by 8 exactly, a single byte 
+segmentation type.
+
+Pixels in an uncompressed ``"BINARY"`` segmentation image are "bit-packed" such
+that 8 pixels are grouped into 1 byte in the stored array. If a given frame
+contains a number of pixels that is not divisible by 8 exactly, a single byte
 will straddle a frame boundary into the next frame if there is one, or the byte
 will be padded with zeroes of there are no further frames. This means that
-retrieving individual frames from segmentation images in which each frame
-size is not divisible by 8 becomes problematic. No further compression may be
-applied to frames of ``"BINARY"`` segmentation images.
+retrieving individual frames from segmentation images in which each frame size
+is not divisible by 8 becomes problematic. For this reason, as well as for
+space efficiency (sparse segmentations tend to compress very well), we
+therefore strongly recommend using ``"JPEG2000Lossless"`` compression with
+``"BINRARY"`` segmentations. This is the only compression method currently
+supported for ``"BINARY"`` segmentations. However, beware that reading these
+single-bit JPEG 2000 images may not be supported by all other tools and
+viewers.
 
 Pixels in ``"FRACTIONAL"`` segmentation images may be compressed using one of
 the lossless compression methods available within DICOM. Currently *highdicom*
 supports the following compressed transfer syntaxes when creating
 ``"FRACTIONAL"`` segmentation images: ``"RLELossless"``,
 ``"JPEG2000Lossless"``, and ``"JPEGLSLossless"``.
 
-Note that there may be advantages to using ``"FRACTIONAL"`` segmentations to
-store segmentation images that are binary in nature (i.e. only taking values 0
-and 1):
-
-- If the segmentation is very simple or sparse, the lossless compression methods
-  available in ``"FRACTIONAL"`` images may be more effective than the
-  "bit-packing" method required by ``"BINARY"`` segmentations.
-- The clear frame boundaries make retrieving individual frames from
-  ``"FRACTIONAL"`` image files possible.
-
 Multiprocessing
 ---------------
 

pyproject.toml

@@ -7,7 +7,7 @@ name = "highdicom"
 dynamic = ["version"]
 description = "High-level DICOM abstractions."
 readme = "README.md"
-requires-python = ">=3.6"
+requires-python = ">=3.10"
 authors = [
     { name = "Markus D. Herrmann" },
 ]
@@ -24,10 +24,6 @@ classifiers = [
     "Operating System :: Microsoft :: Windows",
     "Operating System :: POSIX :: Linux",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.6",
-    "Programming Language :: Python :: 3.7",
-    "Programming Language :: Python :: 3.8",
-    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
@@ -36,21 +32,21 @@ classifiers = [
 ]
 dependencies = [
     "numpy>=1.19",
-    "pillow-jpls>=1.0",
     "pillow>=8.3",
-    "pydicom>=2.3.0,!=2.4.0",
+    "pydicom>=3.0.1",
+    "pyjpegls>=1.0.0",
 ]
 
 [project.optional-dependencies]
 libjpeg = [
-    "pylibjpeg-libjpeg>=1.3",
-    "pylibjpeg-openjpeg>=1.2",
-    "pylibjpeg>=1.4",
+    "pylibjpeg-libjpeg>=2.1",
+    "pylibjpeg-openjpeg>=2.0.0",
+    "pylibjpeg>=2.0",
 ]
 test = [
     "mypy==0.971",
-    "pytest==7.1.2",
-    "pytest-cov==3.0.0",
+    "pytest==7.4.4",
+    "pytest-cov==4.1.0",
     "pytest-flake8==1.1.1",
     "numpy-stubs @ git+https://github.com/numpy/numpy-stubs@201115370a0c011d879d69068b60509accc7f750",
 ]
@@ -68,12 +64,15 @@ documentation = "https://highdicom.readthedocs.io/"
 repository = "https://github.com/ImagingDataCommons/highdicom.git"
 
 [tool.pytest.ini_options]
-addopts = "--doctest-modules"
+minversion = "7"
+addopts = ["--doctest-modules", "-ra", "--strict-config", "--strict-markers"]
 testpaths = ["tests"]
 log_cli_level = "INFO"
+xfail_strict = true
 
 [tool.mypy]
 warn_unreachable = true
+enable_error_code = ["ignore-without-code", "redundant-expr", "truthy-bool"]
 
 [[tool.mypy.overrides]]
 module = "mypy-pydicom.*"

src/highdicom/base.py

@@ -139,8 +139,6 @@ def __init__(
                 "Big Endian transfer syntaxes are retired and no longer "
                 "supported by highdicom."
             )
-        self.is_little_endian = True  # backwards compatibility
-        self.is_implicit_VR = transfer_syntax_uid.is_implicit_VR
 
         # Include all File Meta Information required for writing SOP instance
         # to a file in PS3.10 format.
@@ -154,7 +152,6 @@ def __init__(
             '1.2.826.0.1.3680043.9.7433.1.1'
         )
         self.file_meta.ImplementationVersionName = f'highdicom{__version__}'
-        self.fix_meta_info(enforce_standard=True)
         with BytesIO() as fp:
             write_file_meta_info(fp, self.file_meta, enforce_standard=True)
             self.file_meta.FileMetaInformationGroupLength = len(fp.getvalue())

src/highdicom/content.py

@@ -11,7 +11,7 @@
 from pydicom.sr.coding import Code
 from pydicom.sr.codedict import codes
 from pydicom.valuerep import DS, format_number_as_ds
-from pydicom._storage_sopclass_uids import SegmentationStorage
+from pydicom.uid import SegmentationStorage
 
 from highdicom.enum import (
     CoordinateSystemNames,