Weird JPEG dicoms from ultrasound modalities

[naterichman]

I've been working with quite a bit of Ultrasound data, and in that I've been running into dicoms that have very weird JPEG issues. Which seems to break all major dicom implementations that I've tried (dcmtk, gdcm, pydicom, dicom-rs, I didn't try dcm4che bc I don't use java) and I wanted to start a discussion about how to handle these since I find them fairly regularly in the Ultrasound community...

I've recreated a particular dicom that reproduces these issues since the original data had burned in PHI. The dicom has the following:

• Incorrect NumberOfFrames
  • Both the NumberOfFrames attribute and the length of the BOT in the pixel data point to there being 4 frames, but there are actually only 3 frames
• Frame boundaries fall within a fragment

I created this dicom with a pydicom script (attaching jupyter notebook). But this seems to replicate the issues I've been seeing:

1. In dicom-toimage has these failures when doing each frame

➜  ~/Downloads $ dicom-toimage ~/Downloads/test.dcm --frame 0 --out test0.png
2024-05-14T15:39:12.894779Z ERROR dicom_toimage: failed to decode pixel data

Caused by these errors (recent errors listed first):
  1: Could not decode pixel data
  2: JPEG decoder failure
  3: failed to fill whole buffer

NOTE: Some redundant information has been removed. Set SNAFU_RAW_ERROR_MESSAGES=1 to disable this behavior.
. 
➜  ~/Downloads $ dicom-toimage ~/Downloads/test.dcm --frame 1 --out test1.png
➜  ~/Downloads $ dicom-toimage ~/Downloads/test.dcm --frame 2 --out test2.png
➜  ~/Downloads $ dicom-toimage ~/Downloads/test.dcm --frame 3 --out test3.png
thread 'main' panicked at /home/naterichman/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dicom-transfer-syntax-registry-0.7.0/src/adapters/jpeg.rs:223:16:
range end index 2880000 out of range for slice of length 1440000
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
➜  ~/Downloads $

2. dcmdjpeg fails on this file:

➜  ~/Downloads $ dcmdjpeg test.dcm out.dcm
F: Not a JPEG file: starts with 0xa2 0x8a: decompressing file: test.dcm

So my question is what to do about these files? Is there any interest in trying to support them/fix them?

[naterichman]

Hopefully this link to share works, onedrive link

[Enet4]

Well, that's a handful! :) I've heard before of scenarios where frame boundaries were within a fragment (despite that not being allowed), but to the point of passing the wrong number of frames, it becomes a more severe case of non-compliance.

The current JPEG pixel data decoder is already prepared to read multiple frames, even when there is more than one frame in a fragment, so long as each one doesn't sit in more than one fragment. Even if it were made more robust by handling the presence of unconstrained frame boundaries, then we'd have the case of finding less frames than expected. Maybe there's a case for treating this as a mistake in the attribute rather than a hard error, but this means that we'd need to inspect the pixel data before trusting the Number of Frames attribute, which is often taken in consideration before reading the pixel data. For the time being, you may be better off building a custom tool (via dicom-object for example) which reorganizes the pixel data of those files into one fragment per frame and fixes the number of frames.

---

Out of curiosity, the dcm2jpg tool from dcm4che appears to work... as long as I don't pass a specific frame? Go figure! :)

❯ dcm2jpg.bat test.dcm test.jpg
18:18:02.752 DEBUG - Decompressor: org.dcm4che3.opencv.NativeImageReader
18:18:02.752 DEBUG - Start decompressing frame #1
mai 17, 2024 6:18:02 PM org.opencv.osgi.OpenCVNativeLoader init
INFO: Successfully loaded OpenCV native library.
18:18:02.819 DEBUG - Finished decompressing frame #1
test.dcm -> test.jpg

❯ dcm2jpg.bat --frame=1 test.dcm test.jpg
18:18:16.949 DEBUG - Decompressor: org.dcm4che3.opencv.NativeImageReader
Failed to convert test.dcm: BulkData length limited to -1..2^32-2 but was -8
java.lang.IllegalArgumentException: BulkData length limited to -1..2^32-2 but was -8
        at org.dcm4che3.data.BulkData.setLength(BulkData.java:291)
        at org.dcm4che3.imageio.plugins.dcm.DicomImageReader.generateOffsetLengths(DicomImageReader.java:829)      
        at org.dcm4che3.imageio.plugins.dcm.DicomImageReader.initPixelDataIIS(DicomImageReader.java:803)
        at org.dcm4che3.imageio.plugins.dcm.DicomImageReader.readMetadata(DicomImageReader.java:784)
        at org.dcm4che3.imageio.plugins.dcm.DicomImageReader.read(DicomImageReader.java:445)
        at org.dcm4che3.tool.dcm2jpg.Dcm2Jpg.readImageFromImageInputStream(Dcm2Jpg.java:431)
        at org.dcm4che3.tool.dcm2jpg.Dcm2Jpg.convert(Dcm2Jpg.java:425)

[naterichman]

better off building a custom tool (via dicom-object for example) which reorganizes the pixel data of those files into one fragment per frame and fixes the number of frames.

This is probably what I'll end up doing, thanks! Related question, since one of the "frames" in this image is actually just 2 images squished together, I was curious about how consistent the number of fragments per frame needs to be. I can imagine a scenario in a multiframe image where one image is larger than the others after decompression, but is it allowed for that frame to have more fragments than the other frames?

A note in part 5 sect 8.2 says

Depending on the Fragmentable Encapsulated Transfer Syntax, a frame may be entirely contained within a single fragment, or may span multiple fragments to support buffering during compression or to avoid exceeding the maximum size of a fixed length fragment. A recipient can detect fragmentation of frames by comparing the number of fragments (the number of Items minus one for the Basic Offset Table) with the number of frames. Some performance optimizations may be available to a recipient in the absence of fragmentation of frames, but an implementation that fails to support such fragmentation does not conform to the Standard.

I haven't found it explicitly stated that the number of fragments per frame needs to be the same for all frames but the fact that they say fragmentation can be determined by comparing total fragments to total frames makes me think that is assumed.

Do you happen to know if thats stated anywhere that I'm missing, and or have you run into a dicom with that issue?

[Enet4]

Chaotic as it may be, I believe that a fragment holding two frames (potentially with other fragments holding a different number of frames) is valid.

To the best of my knowledge (both from consulting PS3.5 8.2 and speaking with DICOM WG2 folks on this matter a while back), the constraints regarding the encoding of frames in fragments are that, unless the transfer syntax itself imposes additional constraints, a frame can span multiple whole fragments, and a fragment can contain multiple frames. The only case which preferably should not happen is a frame spanning multiple fragments and its boundaries not matching the boundaries of the fragments.

I say "preferably" because non-conformity with the standard happens in practice, and whether to accept a certain level of non-conformity is a matter of how important it is to integrate with that device against the added complexity, or whether alternatives are available. For example, there is a chance that changing the proposed transfer syntaxes at the device could fix or circumvent the issue.