Is it possible to set a Dataset property value in .dot notation with a dynamic property name ? JSON -> DataSet

[sscotti]

I'd like to be able to create a Dataset, specifically a Modality Worklist File (.wl) from an "order" in JSON format. As an example shown below. This is a nice format that is simple to deal with because it uses the short names from the dicom dictionary and omits the vr type. I'd like to be able to iterate through that dataset and create a .wl (DICOM file) from that and then save it to a folder on my filesystem. If I had static propertynames I could just use the '.dot' notation, but the names are dynamic as I iterate through the list, and for sequences the property values will be of type 'list', rather than 'str', which is fine because I'll have to recurse through those somehow to account for sequences. I noticed that Objects of type PersonName are sort of a special case through some other code that I wrote.

# FUNCTION TO HANDLE SPECIAL CASES, LIKE FOR PersonName Object      
    def safe_serialize(obj):
      default = lambda o: o.family_name + '^' + o.given_name + '^' + o.middle_name + '^' + o.name_prefix + '^' + o.name_suffix
      return json.dumps(obj, default=default) 

So, my question is, how can I use the dot notation with dynamic names to create a dataset from my JSON file description ?

I did find this on SO, but not sure that applies here:

https://stackoverflow.com/questions/14678097/specifying-object-property-in-dot-notation-with-a-variable

The relevant add options from the pydicom documentation are:

Add an element to the Dataset (for elements in the DICOM dictionary):

ds = Dataset()
ds.PatientName = "CITIZEN^Joan"
ds.add_new(0x00100020, 'LO', '12345')
ds[0x0010, 0x0030] = DataElement(0x00100030, 'DA', '20010101')

Add a sequence element to the Dataset

ds.BeamSequence = [Dataset(), Dataset(), Dataset()]
ds.BeamSequence[0].Manufacturer = "Linac, co."
ds.BeamSequence[1].Manufacturer = "Linac and Sons, co."
ds.BeamSequence[2].Manufacturer = "Linac and Daughters, co."

Add private elements to the Dataset

block = ds.private_block(0x0041, 'My Creator', create=True)
block.add_new(0x01, 'LO', '12345')

{
	"Modality": "MR",
	"Allergies": "",
	"PatientID": "CM0000001",
	"Occupation": "Occupation",
	"PatientSex": "M",
	"PatientName": "Person^Test1^",
	"PatientSize": "",
	"ImageComments": "Tech:  SP",
	"MedicalAlerts": "",
	"OperatorsName": "Tech:  SP",
	"PatientWeight": "",
	"PatientAddress": "^^^^^",
	"AccessionNumber": "CMACC00000002",
	"PatientComments": "PatientComments",
	"PatientBirthDate": "20010101",
	"StudyDescription": "MRI BRAIN / BRAIN STEM - WITHOUT CONTRAST",
	"StudyInstanceUID": "",
	"SpecificCharacterSet": "ISO_IR 192",
	"ReferringPhysicianName": "^^^^^",
	"ReferencedStudySequence": [],
	"AdditionalPatientHistory": "test",
	"PatientTelecomInformation": "^^^",
	"AdmittingDiagnosesDescription": "",
	"RequestedProcedureDescription": "MRI BRAIN / BRAIN STEM - WITHOUT CONTRAST",
	"RequestedProcedureCodeSequence": [
		{
			"CodeValue": "70551",
			"CodeMeaning": "[\"70551\"]",
			"CodingSchemeDesignator": "CPT4"
		}
	],
	"ScheduledProcedureStepSequence": [
		{
			"Modality": "MR",
			"AccessionNumber": "CMACC00000002",
			"RequestedProcedureID": "0001",
			"ScheduledStationAETitle": "",
			"ScheduledProcedureStepID": "0001",
			"ScheduledProtocolCodeSequence": [
				{
					"CodeValue": "70551",
					"CodeMeaning": "",
					"CodingSchemeDesignator": "CPT4"
				}
			],
			"ScheduledProcedureStepStartDate": "20210704",
			"ScheduledProcedureStepStartTime": "110000",
			"ScheduledProcedureStepDescription": "MRI BRAIN / BRAIN STEM - WITHOUT CONTRAST"
		}
	],
	"ReferringPhysicianIdentificationSequence": [
		{
			"InstitutionName": "",
			"PersonTelephoneNumbers": "",
			"PersonIdentificationCodeSequence": [
				{
					"CodeValue": "",
					"CodeMeaning": "Local Code",
					"CodingSchemeDesignator": "L"
				}
			]
		}
	]
}

[sscotti]

This actually sort of works, just need to have it handle sequences where sample is my sample dataset above. setattr(mwlDataSet, key, value)

mwlDataSet = Dataset()

for key, value in sample.items():
    print(key)
    print(value)
    if (isinstance(value, str)):
        print(key)
        print(value)
        setattr(mwlDataSet, key, value)
    else: # must be a list or sequence

[sscotti]

This might actually work: Takes in a MWL or just Gernally DICOM File description in JSON format using ShortNames and Values, and a blank dataset, and then returns a Dataset, using recursion for sequences:

def getMWLFromJSON(MWLDict, DataSet):
    
    for key, value in MWLDict.items():
        if (isinstance(value, str)):
            setattr(DataSet, key, value)
        else: # must be a list or sequence
            # setattr(mwlDataSet, key, Dataset())
            sequence = []
            # print(len(value))
            # Create the Sequence Blank Dataset
            for i in range(len(value)):
                sequenceSet = Dataset()
                sequenceSet = getMWLFromJSON(value[i], sequenceSet)
                sequence.append(sequenceSet)
            setattr(DataSet, key, sequence)

    return DataSet

I'll have to test further, but the output I am getting with the test JSON as follows, some data removed. I'll keep working on that, but looks like as easy way to create a MWL from just a JSON description as in the example.

python_mpps_1         | (0008, 0005) Specific Character Set              CS: 'ISO_IR 192'
python_mpps_1         | (0008, 0050) Accession Number                    SH: 'CMACC00000002'
python_mpps_1         | (0008, 0060) Modality                            CS: 'MR'
python_mpps_1         | (0008, 0090) Referring Physician's Name          PN: ''
python_mpps_1         | (0008, 0096)  Referring Physician Identification Sequence  1 item(s) ---- 
python_mpps_1         |    (0008, 0080) Institution Name                    LO: '
python_mpps_1         |    (0040, 1101)  Person Identification Code Sequence  1 item(s) ---- 
python_mpps_1         |       (0008, 0100) Code Value                          SH: '0001'
python_mpps_1         |       (0008, 0102) Coding Scheme Designator            SH: 'L'
python_mpps_1         |       (0008, 0104) Code Meaning                        LO: 'Local Code'
python_mpps_1         |       ---------
python_mpps_1         |    (0040, 1103) Person's Telephone Numbers          LO: ''
python_mpps_1         |    ---------
python_mpps_1         | (0008, 1030) Study Description                   LO: 'MRI BRAIN / BRAIN STEM - WITHOUT CONTRAST'
python_mpps_1         | (0008, 1070) Operators' Name                     PN: 'Tech:'
python_mpps_1         | (0008, 1080) Admitting Diagnoses Description     LO: ''
python_mpps_1         | (0008, 1110)  Referenced Study Sequence  0 item(s) ---- 
python_mpps_1         | (0010, 0010) Patient's Name                      PN: 'Person^Test1^'
python_mpps_1         | (0010, 0020) Patient ID                          LO: 'CM0000001'
python_mpps_1         | (0010, 0030) Patient's Birth Date                DA: '20010101'
python_mpps_1         | (0010, 0040) Patient's Sex                       CS: 'M'
python_mpps_1         | (0010, 1020) Patient's Size                      DS: ''
python_mpps_1         | (0010, 1030) Patient's Weight                    DS: ''
python_mpps_1         | (0010, 1040) Patient's Address                   LO: ''
python_mpps_1         | (0010, 2000) Medical Alerts                      LO: ''
python_mpps_1         | (0010, 2110) Allergies                           LO: ''
python_mpps_1         | (0010, 2155) Patient's Telecom Information       LT: ''
python_mpps_1         | (0010, 2180) Occupation                          SH: 'Occupation'
python_mpps_1         | (0010, 21b0) Additional Patient History          LT: 'test'
python_mpps_1         | (0010, 4000) Patient Comments                    LT: 'PatientComments'
python_mpps_1         | (0020, 000d) Study Instance UID                  UI: 1.3.6.1.4.1.56016.1.1.1.48.1625271194
python_mpps_1         | (0020, 4000) Image Comments                      LT: 'Tech:  SP'
python_mpps_1         | (0032, 1060) Requested Procedure Description     LO: 'MRI BRAIN / BRAIN STEM - WITHOUT CONTRAST'
python_mpps_1         | (0032, 1064)  Requested Procedure Code Sequence  1 item(s) ---- 
python_mpps_1         |    (0008, 0100) Code Value                          SH: '70551'
python_mpps_1         |    (0008, 0102) Coding Scheme Designator            SH: 'CPT4'
python_mpps_1         |    (0008, 0104) Code Meaning                        LO: '["70551"]'
python_mpps_1         |    ---------
python_mpps_1         | (0040, 0100)  Scheduled Procedure Step Sequence  1 item(s) ---- 
python_mpps_1         |    (0008, 0050) Accession Number                    SH: 'CMACC00000002'
python_mpps_1         |    (0008, 0060) Modality                            CS: 'MR'
python_mpps_1         |    (0040, 0001) Scheduled Station AE Title          AE: ''
python_mpps_1         |    (0040, 0002) Scheduled Procedure Step Start Date DA: '20210704'
python_mpps_1         |    (0040, 0003) Scheduled Procedure Step Start Time TM: '110000'
python_mpps_1         |    (0040, 0007) Scheduled Procedure Step Descriptio LO: 'MRI BRAIN / BRAIN STEM - WITHOUT CONTRAST'
python_mpps_1         |    (0040, 0008)  Scheduled Protocol Code Sequence  1 item(s) ---- 
python_mpps_1         |       (0008, 0100) Code Value                          SH: '70551'
python_mpps_1         |       (0008, 0102) Coding Scheme Designator            SH: 'CPT4'
python_mpps_1         |       (0008, 0104) Code Meaning                        LO: '["70551"]'
python_mpps_1         |       ---------
python_mpps_1         |    (0040, 0009) Scheduled Procedure Step ID         SH: '0001'
python_mpps_1         |    (0040, 1001) Requested Procedure ID              SH: '0001'
python_mpps_1         |    ---------

[sscotti]

This is somewhat related to my initial question. My little function seems to work well, but I'm having an issue with the PregnancyStatus Tag, not so much in creating the file, but I think with the encoding since the VR type is US (unsigned int ?16). I am using Orthanc MWL Plug-in to read the created .wl MWL files and it doesn't like '0004' as a string, or 4 as in int, so I'm sure it has something to do with the way I am trying to encode that value. The standard says enumerated value, 0001-0004 as US int.

PHP has a method to return what might be the right value:

private static function uInt16($i, $endianness=false) {

    $f = is_int($i) ? "pack" : "unpack";

    if ($endianness === true) {  // big-endian
        $i = $f("n", $i);
    }
    else if ($endianness === false) {  // little-endian
        $i = $f("v", $i);
    }
    else if ($endianness === null) {  // machine byte order
        $i = $f("S", $i);
    }

    return is_array($i) ? $i[1] : $i;
}

But that encodes it as:

(0010, 21c0) Pregnancy Status US: '\x04\x00'

in the MWL file, and Orthanc is throwing an error when it tries to read that because it is not the correct datatype, or I think so. In my JSON request, I just pass "PregnancyStatus": self::uInt16(4), where I'm using:

$this->PregnancyStatus = self::uInt16(4);

For now, I am just excluding that from my MWL files, but I would like to at least allow that as an option. Maybe I have to pass.

Hopefully just a formatting issue and easy fix ?

[darcymason]

I'm not familiar with MWL files - are they just a type of binary DICOM file like pydicom produces?

A VR of US is binary, and therefore strings wouldn't be the right approach. Setting an integer number in pydicom should work in this case, the output should be encoded as binary, accounting for endianness, similar to the php example you showed. If this isn't working, we'd need some more info to help debug - if pydicom is encoding correctly, then maybe the problem is on Orthanc; can you check with another reader?

Or, is it possible that the endianness is wrong somewhere, so instead of e.g. 4, Orthanc is seeing it as 4 in the high byte (4*256), and rejecting it because that is not in the enumerated values? If you tried setting 1024, that would reverse the bytes and perhaps confirm that that is the problem. If not, please post the specific error message from Orthanc.

[sscotti]

Thank you for responding again.

MWL Files I think are basically like .dcm files, but maybe do not need the Meta Header data. I was creating them from a .txt template and then using DCMTK's dump2dcm to create a .wl file from the text template. That is a little cumbersome, and I was also using some HL7 with that. That is why I'm moving to using a JSON request and pydicom. As an example, this is a sample JSON request. These are the Tags for the MWL file, without the PregnancyStatus Tag though.

{
	"AccessionNumber": "CMACC00000006",
	"AdditionalPatientHistory": "test",
	"AdmittingDiagnosesDescription": "C25.0,F45.41",
	"Allergies": "None in Record",
	"ImageComments": "Tech:",
	"InstitutionName": "Medical",
	"LastMenstrualDate": "",
	"MedicalAlerts": "",
	"Modality": "MR",
	"Occupation": "",
	"OperatorsName": "Tech^",
	"PatientBirthDate": "19571116",
	"PatientComments": "",
	"PatientID": "CM0000002",
	"PatientName": "Last^First^",
	"PatientSex": "M",
	"PatientSize": "",
	"PatientWeight": "",
	"ReferringPhysicianIdentificationSequence": [{
		"InstitutionName": "Medical",
		"PersonIdentificationCodeSequence": [{
			"CodeMeaning": "Local Code",
			"CodeValue": "0002",
			"CodingSchemeDesignator": "L"
		}],
		"PersonTelephoneNumbers": "^WPN^CP^"
	}],
	"ReferringPhysicianName": "0001:Last^First^",
	"ReferencedStudySequence": [],
	"RequestedProcedureID": "0005",
	"RequestedProcedureDescription": "MRI C-SPINE WITH CONTRAST",
	"ScheduledProcedureStepSequence": [{
		"Modality": "MR",
		"ScheduledProcedureStepDescription": "MRI C-SPINE WITH CONTRAST",
		"ScheduledProcedureStepID": "0005",
		"ScheduledProcedureStepStartDate": "20210719",
		"ScheduledProcedureStepStartTime": "010000",
		"ScheduledProtocolCodeSequence": [{
			"CodeMeaning": "[\"72142\",\"A9579\"]",
			"CodeValue": "0005",
			"CodingSchemeDesignator": "C4"
		}],
		"ScheduledStationAETitle": "AET",
		"RequestedProcedureID": "0005"
	}],
	"SpecificCharacterSet": "ISO_IR 192",
	"StudyDescription": "MRI C-SPINE WITH CONTRAST",
	"StudyInstanceUID": "1.3.6.1.4.1.56016.1.1.1.50.1627832246"
}

My Python code runs as a "Plug-In" within Orthanc and uses mostly these to generate the .wl file:

MWLFromJSONCreateAndSave grabs the JSON data from the request and then does some stuff to create the .wl file, including a call to getMWLFromJSON(query, dataset).

def MWLFromJSONCreateAndSave(output, uri, **request):

    if request['method'] != 'POST':
        output.SendMethodNotAllowed('POST')
    else:
        query = json.loads(request['body'])
        print(json.dumps(query))
        if not os.path.exists(WORKLIST_DIR):
            os.makedirs(WORKLIST_DIR)
        response = dict()
        print("Making MWL from JSON")
        dataset = Dataset()
        dataset = getMWLFromJSON(query, dataset)
        dataset.is_little_endian = True # 'Dataset.is_little_endian' and 'Dataset.is_implicit_VR' must be set appropriately before saving
        dataset.is_implicit_VR = True
        # Set creation date/time
        dt = datetime.now()
        dataset.ContentDate = dt.strftime('%Y%m%d')
        timeStr = dt.strftime('%H%M%S.%f')  # long format with micro seconds
        dataset.ContentTime = timeStr
        response = SaveDatasetDB(query, dataset)
        print("Writing test file", response['filename'])
        dataset.save_as(WORKLIST_DIR + response['filename'] ,write_like_original=True) # True takes care of not explcitly setting (0002, 0002) MediaStorageSOPClassUID & (0002, 0003) MediaStorageSOPInstanceUID
        DB = ((".  Saved to PACS DB " + RISDB['MWLdatabase'])  if  response['DB'] else ".  Error Saving to Orthanc RIS")
        response['status'] = "MWL:  " + response['filename'] + DB
        output.AnswerBuffer(json.dumps(response, indent = 3), 'application/json')

orthanc.RegisterRestCallback('/mwl/create_from_json', MWLFromJSONCreateAndSave)

def getMWLFromJSON(MWLDict, DataSet):
    
    for key, value in MWLDict.items():
        if (isinstance(value, str) or isinstance(value, int)):
            setattr(DataSet, key, value)
        else: # must be a list or sequence
            # setattr(mwlDataSet, key, Dataset())
            sequence = []
            # Create the Sequence Blank Dataset
            for i in range(len(value)):
                sequenceSet = Dataset()
                sequenceSet = getMWLFromJSON(value[i], sequenceSet)
                sequence.append(sequenceSet)
            setattr(DataSet, key, sequence)
    return DataSet

You could actually probably remove some of my extra stuff and actually create a .wl file from just these items.

Pretty sure the issue is related to the byte order for the PregnancyStatus value or for the file itself in some way.

[sscotti]

Revisited this after quite a hiatus. This might actually be better because it can handle nested sequence and some other issues. Still need to really test it:

def getMWLFromJSON(MWLDict, DataSet):
    for key, value in MWLDict.items():
        if isinstance(value, str) or isinstance(value, int):
            setattr(DataSet, key, value)
        elif isinstance(value, list):
            # handle nested sequence
            sequence = []
            for i in range(len(value)):
                sequenceSet = Dataset()
                sequenceSet = getMWLFromJSON(value[i], sequenceSet)
                sequence.append(sequenceSet)
            setattr(DataSet, key, sequence)
        elif key in DataSet:
            # handle multiple values for same attribute
            existing_value = getattr(DataSet, key)
            if not isinstance(existing_value, list):
                existing_value = [existing_value]
            existing_value.append(value)
            setattr(DataSet, key, existing_value)
        else:
            # handle non-standard attribute
            DataSet.add_new(key, value)
    return DataSet

Interesting that ChatGPT 4 gives a pretty decent verbal description of what it does.

The function getMWLFromJSON takes two input arguments: MWLDict and DataSet. The MWLDict is a dictionary containing data in a JSON-like structure. The DataSet is an instance of a custom class where you want to store the data from the JSON-like structure.

The function iterates through the key-value pairs in the MWLDict dictionary. If the value is a string or an integer, it directly sets the attribute with the given key and value in the DataSet instance.

If the value is a list, it handles the nested sequence. For each element in the list, it creates a new instance of the Dataset class and calls the getMWLFromJSON function recursively to process the element. It then appends the processed element to the sequence list and sets the attribute with the given key and sequence in the DataSet instance.

If the key already exists in the DataSet instance, it handles the case of having multiple values for the same attribute. It checks if the existing value is a list; if not, it converts it into a list. It then appends the new value to the existing values and sets the attribute with the given key and the updated list of values in the DataSet instance.

If none of the above cases apply, it handles the non-standard attribute by calling the add_new method of the DataSet instance with the given key and value.

Finally, the function returns the updated DataSet instance.

[sscotti]

Thanks. I have not completely tested it, but it does seem to work with my sample JSON object. That is a really convenient way to create a dataset from just simple JSON. The .to_json() produces something similar for JSON, but it explicitly includes the vr's and Values. For my application, I'd like to use the simpler JSON noted above to create a DataSet and then write it to the filesystem.

BTW, not sure you use Docker much. I have some Orthanc instances running with my MPPS/MWL container (Python container), and I tried sharing a mount / bind between the 2 containers. I cannot write the dataset, e.g. mwlDataSet.save_as('/MWLFiles2/test.wl') to the shared bind, but I can write to a bind that is only used by the MPPS/MWL container.

Not sure if that is related to my Docker setup or something to do with the Orthanc Instance locking that folder. I get a permissions error when trying to write to the shared folder, and I get a little "lock" symbol showing up on my Mac OS Host. Could be all kinds of issues really since Mac OS has some more complicated permissions of late.

I pulled some stuff from you documentation to write the file and to set the MetaHeader. I need to correctly set the Header Info to the correct data for MRImages, or pull them from the DataSet, although I think MWL files don't actually need the MetaHeader. A dcmdump of the file that is successfully written seems to check out. Is there a way to automatically set the Meta header by just reading the modality from the DataSet ?

# Dicom-Meta-Information-Header
# Used TransferSyntax: Little Endian Explicit
(0002,0002) UI =CTImageStorage                          #  26, 1 MediaStorageSOPClassUID
(0002,0003) UI [1.2.3]                                  #   6, 1 MediaStorageSOPInstanceUID
(0002,0012) UI [1.2.3.4]                                #   8, 1 ImplementationClassUID

    print("Setting file meta information...")
    # Populate required values for file meta information
    file_meta = FileMetaDataset()
    file_meta.MediaStorageSOPClassUID = '1.2.840.10008.5.1.4.1.1.2'
    file_meta.MediaStorageSOPInstanceUID = "1.2.3"
    file_meta.ImplementationClassUID = "1.2.3.4"
    mwlDataSet.file_meta = file_meta
    # Set the transfer syntax
    mwlDataSet.is_little_endian = True
    mwlDataSet.is_implicit_VR = True
    # Set creation date/time
    dt = datetime.now()
    mwlDataSet.ContentDate = dt.strftime('%Y%m%d')
    timeStr = dt.strftime('%H%M%S.%f')  # long format with micro seconds
    mwlDataSet.ContentTime = timeStr
    print("Writing test file", '/MWLFiles2/test.wl')
    mwlDataSet.save_as('/MWLFiles2/test.wl')
    print("File saved.")

Thanks.