Nodata is not respected when using expression

[dvd3v]

I'm using an expression where I only select pixels of above X value. If the evaluation is true, the original band value is returned and if the evaluation is false the nodata value is returned. The expression looks like:

"where((b1 > 10),b1,-999);". Where -999 is the nodata value.

When rendering the map, the nodata values returned by the expression are not ignored, while the original nodata values are still handled correctly. Even when overwriting the nodata value, the nodata values in the returned expression array are not respected.

The input parameters are:

{
   "url":"s3_url",
   "rescale":"0,100",
   "colormap_name":"terrain",
   "expression":"where((b1 > 10),b1,-999);"
}

Is it possible to handle the -999 values of the expression array as actual nodata values?

[vincentsarago]

The issue here is in rio-tiler. The expression applied after data reading and we have no way to know if the mask was created from nodata / mask / alpha band, which is why we can't update the Mask https://github.com/cogeotiff/rio-tiler/blob/master/rio_tiler/io/cogeo.py#L417-L432

One possible solution is to use the post_process option of rio-tiler Reader (https://cogeotiff.github.io/rio-tiler/readers/#read-options) and to pass it via the dataset_dependency which default to:

from dataclasses import dataclass
import numpy
from enum import Enum
from titiler.core import dependencies
from fastapi import Query

def b1mask(data: numpy.ndarray, mask: numpy.ndarray):
    b1_mask = numpy.where(data[0] > 10, numpy.uint8(255), numpy.uint8(0))
    mask = numpy.all([mask, b1_mask], axis=0).astype(numpy.uint8) * 255
    return data, mask


available_processes = {
    "b1mask": b1mask
}

Process = Enum(  # type: ignore
    "Process", [(a, p) for a, p in available_processes.items()]
)


@dataclass
class DatasetParams(dependencies.DatasetParams):

    post_process: Process = Query(None, alias="p")  # 

    def __post_init__(self):
        """Post Init."""
        super().__post_init__()

        if self.post_process:
            self.post_process = Process.value  # noqa

[dvd3v]

Thanks! This got me in the right direction. However I have some troubles with the Enum, the "b1mask" is not a valid enumeration member when I use it as the input of the p paramater. Any idea what I'm missing? The Enum list looks correct: [('b1mask', <function b1mask at 0x12a1d4820>)]. When I use the post_processing without the Enum the function gets executed and the results are as expected, very awesome!

Furthermore I'm trying to figure out if it is possible to dynamically pass the filter value, instead of hardcoding it in the function. I only seem to be able to pass the post_process parameter, is this correct?

[vincentsarago]

Furthermore I'm trying to figure out if it is possible to dynamically pass the filter value, instead of hardcoding it in the function. I only seem to be able to pass the post_process parameter, is this correct?

yes that's a limitation of the post_process function 😞

to make this configurable you'll need to write more hacky code

from dataclasses import dataclass
import numpy
from enum import Enum
from titiler.core import dependencies
from fastapi import Query
from types import DynamicClassAttribute

class mask:
    def __init__(self, threshold: int = 10):
        self.threshold = threshold

    def __call__(self, data: numpy.ndarray, mask: numpy.ndarray):
        b1_mask = numpy.where(data[0] > self.threshold, numpy.uint8(255), numpy.uint8(0))
        mask = numpy.all([mask, b1_mask], axis=0).astype(numpy.uint8) * 255
        return data, mask


available_processes = {
    "mask": mask
}


class Process(str, Enum):
    """Available Output image type."""

    mask = "mask"

    @DynamicClassAttribute
    def function(self):
        """Return rio-tiler image default profile."""
        return available_processes.get(self._name_, {})


@dataclass
class DatasetParams(dependencies.DatasetParams):

    post_process: Process = Query(None, alias="p")  #
    threshold: int = Query(10)

    # By default all `DatasetParams` attributes will be forwarded to the BaseReader method
    # but threshold is not one of defined options so we need to make sure that it's not passed 
    # when we do `**dataset_params` in the endpoint
    def keys(self):
        """Return Keys."""
        keys = list(self.__dict__.keys())
        return [k for k in keys if k not in ["threshold"]]

    def __post_init__(self):
        """Post Init."""
        super().__post_init__()

        if self.post_process:
            self.post_process = self.post_process.function(self.threshold)  # noqa

[vincentsarago]

@dvd3v next week I'm going to work on something more clean to accept both post_process and user input

[dvd3v]

Thanks for the help! The "hacky" code helps in understanding the inner workings a bit more. I've implemented and extended your example and it works great!

Thanks for all the awesome work on this project, I'm looking forward to your integration of the post_process function with user input.