General question about data acquisition and processing

[D49ll]

Hi,

I am a master student trying to develop a new tracking system for orthopedic surgery/diagnostics. For this project i've got access to a new optical tracking system (OTS) and its SDK. However after some weeks researching and trying to get data from the OTS, I now feel like I am at a loss. I've created this post with the hope of getting some help from the community, which are more experienced in this field.

The SDK uses a master-client relationship. To be able to receive data from the OTS the master has to be running. When its running the master sends messages/data continuously to anyone listening on the specified port, based on what the user specifies. The SDK provides two main functions, SendMaster() and ProcessMessage(), the first lets the user specify what it wants to subscribe to and the second how the data should be processed.

Lets consider the most basic scenario where we only have one tracking probe connected to the system and want to measure the distance from the probe to the camera (the transform). Below i've listed my current approach of integrating the SDK with PlusToolKit (vtkMyDevice.h/cxx). This should only contain the basic functionality, however the goal is to have the end result somewhat resemble the capabilities of NDI's Polaris (which will the system to compare against).

Steps:

1. this->StartThreadForInternalUpdates = true: due to the master sending data continuously
2. InternalConnect(): start the master. Create an instance of my SDKApp and connect the local host with specified port.
3. InternalStartRecording(): Specify with SendMaster that I want the transform from pen to camera.
4. InternatUpdate(): Receives data from master, through localhost port 1390. The data can be parsed using the SDK message system. To begin with the data will only be a 4x4 homogeneous matrix.

This is where I'm stuck. I've read on the "adding a device" page that the data can be stored in different ways, i.e vtkPLusDataSource::AddItem, vtkPlusDevice::AddVideoItemToVideo source etc. But wow is the data being sent from the plustoolkit server? I've seen that it uses OpenIGTLink, but I cant seem to understand how I specify what and how the data should be sent.

The project is suppose to use a HoloLens place a 3D-image (hologram) onto a patient. Where the transforms received from the OTS will enable rigid registration of said hologram. That being said, I first want to use a more simpler solution, like maybe sending the data to 3D slicer or just another client?

Do you have any tips and information as to how I can solve this? All replies will be greatly appreciated!

[adamrankin]

This is where I'm stuck. I've read on the "adding a device" page that the data can be stored in different ways, i.e vtkPLusDataSource::AddItem, vtkPlusDevice::AddVideoItemToVideo source etc. But wow is the data being sent from the plustoolkit server?

There is a lot of infrastructure in place under the hood, so to speak. In a sample configuration file for a tracker:

<Device
      Id="TrackerDevice"
      Type="PolarisTracker"
      ToolReferenceFrame="Tracker" >
      <DataSources>
        <DataSource Type="Tool" Id="Tool" RomFile="NdiToolDefinitions/8700339.rom"  />
        <DataSource Type="Tool" Id="Stylus" RomFile="NdiToolDefinitions/8700340.rom"  />
        <DataSource Type="Tool" Id="Reference" RomFile="NdiToolDefinitions/8700449.rom"  />
      </DataSources>
      <OutputChannels>
        <OutputChannel Id="TrackerStream" >
          <DataSource Id="Tool"/>
...
  <PlusOpenIGTLinkServer 
    MaxNumberOfIgtlMessagesToSend="1"
    MaxTimeSpentWithProcessingMs="50"
    ListeningPort="18944"
    SendValidTransformsOnly="true"
    OutputChannelId="TrackerStream" >
    <DefaultClientInfo>
      <MessageTypes>
        <Message Type="TRANSFORM" />
      </MessageTypes>
      <TransformNames>
        <Transform Name="StylusTipToReference" />
        <Transform Name="ToolToReference" />
...

The data sources for a tracker are usually of type Tool (not video).
In your device, you should ensure that the user has properly configured the device, and that means they should have at least one Tool data source (probably, depends how your tracker operates, but if it's similar to an NDI system, it's one Tool data source per tracked tool).

Thus, you should find that your PlusDevice has a list of 0+ data sources (1+ if configured correctly) that you can dump data into. In your configuration file, if that Tool data source is outputted in the channel, then PlusServer will send that transform to a client.

Really, I would just mimic the key function of NDI tracker. There's some extra complexity to the device, but it's not that bad.

[D49ll]

Thanks for your reply!

After more research it turns out that cannot use the SDK provided due to it being a winapp. So I had to create my own socket functionality, i.e my own SDK, much like what NDI has provided in ndicapi.h/cxx and ndi_socket.h/cxx.

Currently I've only been able to create this client in my own project (not using PlusToolKit) and running in my own threads, however I have not been able to implement it in PlusToolKit framework. This is due to me not fully understanding the developing process needed. My standalone client is wrapped around asio.hpp for socket communication and works seemly ok with the master (server). I am able to establish a connection and send/receive data to/from the master. These messages provides basic functionality like starting/stopping tracking, requesting a measurement etc.

My main issue now is that i cannot figure out how to develop the different internal functions in vtkPlusMyDevice and running them in debug mode. Do I need to build via cmake for every change i do and hope it works?

If its not to much of a ask, is it possible to get some guidance in how you develop and test vtkPlusMyDevice when adding a new device?

Hopefully I was able to clearly portray the issue I'm facing, if not I can try to be even more detailed. I appreciate any tips you might have! As you might tell im quite new to this type of program structure, but i would like to learn :)

[adamrankin]

For debugging I'd recommend using visual studio. But yes, line by line debugging is common.

With regards to how to develop your device, I highly recommend getting a good breakpoint debugging setup and following along how another device functions, when functions are called, etc...

[D49ll]

I think i did a bad job explaining my issues. I am aware of the general idea behind debugging and coding in c++, but im having difficulties understanding the project hieraki. However i feel like the questions i have might be suited in a new seperate post. Ill create one. Thanks for your reply!