Skip to content

03. Implementation

Jump to bottom
Andrew C. Freeman edited this page May 9, 2023 · 6 revisions

Video sources

I've tried to make the ADΔER library generic and extensible, to be able to support a wide variety of camera types. As of now, it can operate on framed video sources and event video sources from iniVation cameras.

To incorporate support for a new camera, you need to program these things:

  • A data reader/decoder for the camera files/stream
  • A scheme for converting the camera's data representation to an intensity and time measurement
  • Call the integrate_for_px function (here) for each intensity measurement
  • Try to parallelize it to make it fast

And that's it! Everything else will handle itself, since we use the same ADΔER pixel model underneath each transcode process.

Framed sources

DVS/DAVIS sources

Mode (i): deblurred frames

Mode (ii): deblurred APS frames + DVS events

Mode (iii): DVS events

Event pixel lists

We will first examine the integration of a single \eventformat{} pixel model. We represent an \eventformat{} pixel state with a linked list. Each node in the list consists of a decimation factor, $D$, an intensity integration measurement, $I$, and a time interval measurement, $\Delta t$. The connections between nodes carry an \eventformat{} event representing the intensity that the parent node integrated \textit{before} the child node was created. When initializing a new list, we set $D$ to be the maximal value possible to represent the first intensity we intend to integrate. To maintain precision, the values $I$ and $\Delta t$ in the list are floating point numbers. Since \eventformat{} is source-agnostic, we interpret the incoming values as generic "intensity units", rather than tying them to a real-world measure of intensity, such as photons, and we can scale the input values of individual data sources according to our needs.

Let's walk through a visual example of this list structure.

Suppose our first intensity to integrate is 101. We initialize our head node with $D=\lfloor\log_2(101)\rfloor = 6$

  • Now, suppose we integrate the intensity 101, spanning 20 time units (ticks). The head only accumulates 4/101$ of the intensity units before saturating its ^D=64$ integration. Thus, the time spanned for this partial integration is $(64/101)\cdot 20 = 12.67$ ticks. We can represent an \eventformat{} event (without its spatial coordinate), illustrated in angle brackets, as the node connection. These events consist of integer numbers, so we use $\lfloor\Delta t \rfloor = 12$. At this stage, we create a child node to represent the remaining integration. The child takes on the head node's $D$ (i.e., $D=6$). We integrate the remaining 01-64=37$ intensity units for the child node, spanning $(37/101)\cdot 20 = 7.33$ ticks. Finally, we increment the head node's $D$ value and integrate the remaining $I=37, \Delta t = 7.33$ intensity.

TODO

TODO

TODO

Clone this wiki locally