Image processing

The image processing pipeline is critical for transforming sensor voltages into display images. The pipeline accounts for the sensor properties and the display properties, linking these two critical hardware devices.

The ISETCam image Processing structure (ip) is organized like the other important ISETCam structures. But in some ways it is profoundly different. Whereas the other main structures are connected to physics, optics, or hardware, the ip is connected only to software. For many ISETCam users the image processing steps in the cameras they use are entirely within their control.

The ip methods in ISETCam are mainly useful as a way to organize and carefully label the critical image processing steps. The ip routines also support many of the image quality metrics people use to measure the spatial and color quality of images.

The key ip routines are the same as for the other structures, ipCreate, ipCompute, ipSet/Get, ipWindow, and ipPlot. We summarize the key concepts of the ip pipeline and the structure here.

For more, see

• Image Processing Methods
• Tutorials and scripts for image processing.

IP Basics

The simulation from a scene spectral radiance, through the optics, to a sensor, and then the image processing pipeline yields a rendered sRGB. The code looks like this:

scene = sceneCreate; scene = sceneSet(scene,'fov',10);
oi = oiCreate('wvf'); oi = oiCompute(oi,scene,'crop',true);
sensor = sensorCreate('imx363'); sensor = sensorSet(sensor,'fov',10,oi); sensor = sensorCompute(sensor,oi);
ip = ipCreate; ip = ipCompute(ip,sensor);

We can visualize the result in the ipWindow

ipWindow(ip);

The key ip parameters are on the right side of the window. Beyond some of the earlier examples, the ip parameters include the names of methods that are used in the image processing. For example, the demosaicking method and the method for converting from sensor space to the internal color space are both shown.

As usual, the image processing parameters can be controlled programmatically with the ipSet and ipGet commands. For example,

>> ipGet(ip,'demosaic method')

ans =

    'bilinear'

Or

>> ipGet(ip,'internal color space')

ans =

    'stockman'
>> ipGet(ip,'internal color space')

ans =

    'stockman'

>> 

There are also many plotting functions and functions for analyzing image quality metrics, including the ISO 12233 standard for the slanted edge, CIELAB, Spatial CIELAB, and noise. Here is an example.

ipPlot(ip,'horizontal line',[1 160]);

For a deeper dive, go to the Image Processing Methods page. There are many examples of using the ip methods on the page of tutorials and scripts for image processing.