Stereo Image Acquisition

[Karl-Philippe]

Hello,

I would like to use Plus to acquire stereo images from a tracked endoscope with Plus (Two video streams and one tracking stream). Is it possible to synchronize two video streams with the virtual mixer?

I can successfully start this server (see configuration below) to acquire two video streams and synchronize them (without the tracking). However, in 3D Slicer (via OpenIGTLink), I can only see the first video stream. Is this the proper way to write the Plus configuration?

<PlusConfiguration version="2.3">
  <DataCollection StartupDelaySec="1.0">
    <DeviceSet 
      Name = "PlusServer: 2 video streams"
      Description="Broadcasting acquired video and tracking streams through OpenIGTLink" />

    <!-- Define Video Device 1 ImageType="RGB_COLOR" -->
    <Device 
      Id="VideoDevice1"
      Type="MmfVideo"
      FrameSize="1920 1080"
      VideoFormat="YUY2"
      CaptureDeviceId="0">
      <DataSources>
        <DataSource
          Type="Video"
          Id="Video1"
          PortUsImageOrientation="UF"/>
      </DataSources>
      <OutputChannels>
        <OutputChannel Id="VideoStream1" VideoDataSourceId="Video1"/>
      </OutputChannels>
    </Device>

    <!-- Define Video Device 1 ImageType="RGB_COLOR" -->
    <Device 
      Id="VideoDevice2"
      Type="MmfVideo"
      FrameSize="1920 1080"
      VideoFormat="YUY2"
      CaptureDeviceId="1">
      <DataSources>
        <DataSource
          Type="Video"
          Id="Video2"
          PortUsImageOrientation="UF"/>
      </DataSources>
      <OutputChannels>
        <OutputChannel Id="VideoStream2" VideoDataSourceId="Video2"/>
      </OutputChannels>
    </Device>


    <!-- Define VirtualMixer device -->
    <Device Id="TrackedVideoDevice" Type="VirtualMixer">
      <InputChannels>
        <InputChannel Id="VideoStream1" />
        <InputChannel Id="VideoStream2" />
      </InputChannels>
      <OutputChannels>
        <OutputChannel Id="TrackedVideoStream" />
      </OutputChannels>
    </Device>

    <!-- Define VirtualCapture device -->
    <Device
      Id="CaptureDevice"
      Type="VirtualCapture"
      BaseFilename="OpenIGTTrackedVideoRecordingTest.igs.mha"
      EnableCapturingOnStart="FALSE" >
      <InputChannels>
        <InputChannel Id="TrackedVideoStream" />
      </InputChannels>
    </Device>

  </DataCollection>

  <CoordinateDefinitions>
    <Transform From="Image1" To="Reference"
      Matrix="
        1.0 0.0 0.0 0.0
        0.0 1.0 0.0 0.0
        0.0 0.0 1.0 0.0        
        0 0 0 1" />
  </CoordinateDefinitions>

  <CoordinateDefinitions>
    <Transform From="Image2" To="Reference"
      Matrix="
        1.0 0.0 0.0 0.0
        0.0 1.0 0.0 0.0
        0.0 0.0 1.0 0.0        
        0 0 0 1" />
  </CoordinateDefinitions>

  <!-- PlusOpenIGTLinkServer definition -->
  <PlusOpenIGTLinkServer 
    MaxNumberOfIgtlMessagesToSend="1" 
    MaxTimeSpentWithProcessingMs="50" 
    ListeningPort="18944" 
    SendValidTransformsOnly="true" 
    OutputChannelId="TrackedVideoStream" >
    <DefaultClientInfo>
      <MessageTypes>
        <Message Type="IMAGE" />
      </MessageTypes>
      <ImageNames>
        <Image Name="Image1" EmbeddedTransformToFrame="Reference"/>
        <Image Name="Image2" EmbeddedTransformToFrame="Reference"/>
      </ImageNames>
    </DefaultClientInfo>
  </PlusOpenIGTLinkServer>
</PlusConfiguration>

Thanks,

[Sunderlandkyl]

Channels can only contain one video stream. You will need to keep the channels separate and use a different PlusOpenIGTLinkServer for each.

[Karl-Philippe]

Thank you, I am able to successfully acquire two streams on two separate PlusOpenIGTLinkServer and acquire them in 3D Slicer. However, since each of the servers is acquired with two separate IGTLConnector, it is not possible to record both channels (servers) simultaneously. Is there a way to sync those two PlusOpenIGTLinkServer and acquire them with one IGTLConnector, to then record the two video streams with the Plus Remote Slicer extension?

Thank you,

[Sunderlandkyl]

You can record them both simultaneously in Slicer using the Sequences module. It has the disadvantage of not continuously streaming the images to file, but you can save any number of synchronized images and transforms.

[Karl-Philippe]

Ok thank you it worked. However, what is the best way to save the synchronized sequence nodes? Is there a way to save the synchronized sequences in a single file (i.e. .mha)? For now, I can save the whole scene but it is not very optimal.

[Sunderlandkyl]

You can save the entire scene in one mrb file, but aside from that there is no method currently available that I can think of.

[Karl-Philippe]

Ok thank you for you help!