Skip to content

Rename package and redesign types #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,6 @@ jobs:
- run: |
julia --project=docs -e '
using Documenter: DocMeta, doctest
using FluorophoreColors
DocMeta.setdocmeta!(FluorophoreColors, :DocTestSetup, :(using FluorophoreColors); recursive=true)
doctest(FluorophoreColors)'
using MultiChannelColors
DocMeta.setdocmeta!(MultiChannelColors, :DocTestSetup, :(using MultiChannelColors); recursive=true)
doctest(MultiChannelColors)'
4 changes: 2 additions & 2 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name = "FluorophoreColors"
name = "MultiChannelColors"
uuid = "d4071afc-4203-49ee-90bc-13ebeb18d604"
authors = ["Tim Holy <[email protected]> and contributors"]
version = "0.1.0"
Expand All @@ -13,7 +13,7 @@ Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
Requires = "ae029012-a4dd-5104-9daa-d747884805df"

[compat]
ColorTypes = "0.11.1"
ColorTypes = "0.11.2"
ColorVectorSpace = "0.9"
Colors = "0.12"
FixedPointNumbers = "0.8"
Expand Down
86 changes: 5 additions & 81 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,83 +1,7 @@
# FluorophoreColors
# MultiChannelColors

[![Build Status](https://github.com/JuliaImages/FluorophoreColors.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/JuliaImages/FluorophoreColors.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![Coverage](https://codecov.io/gh/JuliaImages/FluorophoreColors.jl/branch/main/graph/badge.svg)](https://codecov.io/gh/JuliaImages/FluorophoreColors.jl)
[![Build Status](https://github.com/JuliaImages/MultiChannelColors.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/JuliaImages/MultiChannelColors.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![Coverage](https://codecov.io/gh/JuliaImages/MultiChannelColors.jl/branch/main/graph/badge.svg)](https://codecov.io/gh/JuliaImages/MultiChannelColors.jl)
[![](https://img.shields.io/badge/docs-stable-blue.svg)](https://JuliaImages.github.io/MultiChannelColors.jl/stable)

This package defines [color types](https://github.com/JuliaGraphics/ColorTypes.jl) for use with multichannel fluorescence imaging. Briefly, you can specify the intensity of each color channel plus an RGB value associated with the peak emission
wavelength of each fluorophore.

## Basic usage

Perhaps the easiest way to learn the package is by example. Suppose we are imaging two fluorophores, EGFP and tdTomato.

```julia
julia> using FluorophoreColors

julia> channels = (fluorophore_rgb["EGFP"], fluorophore_rgb["tdTomato"])
(RGB{N0f8}(0.0,0.925,0.365), RGB{N0f8}(1.0,0.859,0.0))

julia> ctemplate = ColorMixture{N0f16}(channels)
(0.0N0f16₁, 0.0N0f16₂)
```

This creates an all-zero "template" color object. Note that we've specified the element type, `N0f16`, for 16-bit color depth.
The subscripts `₁` and `₂` are hints that this is not an ordinary tuple; each represents the intensity in the corresponding channel.

We use `ctemplate` to construct any other color:

```julia
julia> c = ctemplate(0.25, 0.75)
(0.25N0f16₁, 0.75N0f16₂)

julia> convert(RGB, c)
RGB{N0f16}(0.75,0.87549,0.09117)
```

The latter is how this color would be rendered in a viewer.

## Overflow protection

Depending on the colors you pick for conversion to RGB (e.g., `channels`), it is possible to exceed the 0-to-1 bounds of RGB.
With the choice above,

```julia
julia> c = ctemplate(0.99, 0.99)
(0.99001N0f16₁, 0.99001N0f16₂)

julia> convert(RGB, c)
ERROR: ArgumentError: component type N0f16 is a 16-bit type representing 65536 values from 0.0 to 1.0,
but the values (0.9900053f0, 1.7664759f0, 0.36105898f0) do not lie within this range.
See the READMEs for FixedPointNumbers and ColorTypes for more information.
Stacktrace:
[...]
```

If you want to guard against such errors, one good choice would be

```julia
julia> convert(RGB{Float32}, c)
RGB{Float32}(0.9900053, 1.7664759, 0.36105898)
```

Conversions to floating-point types also tend to be faster, since the values do not have to be checked.

## Advanced usage

`ctemplate` stores the RGB *values* for each fluorophore as a type-parameter. This allows efficient conversion to RGB
without running into world-age problems that might otherwise arise from auto-generated conversion methods.
However, constructing `ctemplate` as above is an inherently non-inferrable operation. If you want to construct such colors
inferrably, you can use the macro version:

```julia
f(i1, i2) = ColorMixture{N0f16}((fluorophore_rgb"EGFP", fluorophore_rgb"tdTomato"), (i1, i2))
```

Note the absence of `[]` brackets around the fluorophore names.

## Why are the RGB colors encoded in the *type*? Why not a value field?

In many places, JuliaImages assumes that you can convert from one color space to another purely from knowing the type you want to convert to. This would not be possible if the RGB colors were encoded as a second field of the color.

## I wrote some code and got lousy performance. How can I fix it?

To achieve good performance, in some cases the RGB *values* must be aggressively constant-propagated, a feature available only on Julia 1.7 and higher. So if you're experiencing this problem on Julia 1.6, try a newer version.
This package defines [color types](https://github.com/JuliaGraphics/ColorTypes.jl) for use with multichannel fluorescence and hyperspectral imaging. See the [documentation](https://JuliaImages.github.io/MultiChannelColors.jl/stable) for more information.
2 changes: 1 addition & 1 deletion docs/Project.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[deps]
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
FluorophoreColors = "d4071afc-4203-49ee-90bc-13ebeb18d604"
MultiChannelColors = "d4071afc-4203-49ee-90bc-13ebeb18d604"
16 changes: 9 additions & 7 deletions docs/make.jl
Original file line number Diff line number Diff line change
@@ -1,24 +1,26 @@
using FluorophoreColors
using MultiChannelColors
using Documenter

DocMeta.setdocmeta!(FluorophoreColors, :DocTestSetup, :(using FluorophoreColors); recursive=true)
DocMeta.setdocmeta!(MultiChannelColors, :DocTestSetup, :(using MultiChannelColors); recursive=true)

makedocs(;
modules=[FluorophoreColors],
modules=[MultiChannelColors],
authors="Tim Holy <[email protected]> and contributors",
repo="https://github.com/JuliaImages/FluorophoreColors.jl/blob/{commit}{path}#{line}",
sitename="FluorophoreColors.jl",
repo="https://github.com/JuliaImages/MultiChannelColors.jl/blob/{commit}{path}#{line}",
sitename="MultiChannelColors.jl",
format=Documenter.HTML(;
prettyurls=get(ENV, "CI", "false") == "true",
canonical="https://JuliaImages.github.io/FluorophoreColors.jl",
canonical="https://juliaimages.org/MultiChannelColors.jl",
assets=String[],
),
pages=[
"Home" => "index.md",
"FAQ" => "faq.md",
"Reference" => "api.md",
],
)

deploydocs(;
repo="github.com/JuliaImages/FluorophoreColors.jl",
repo="github.com/JuliaImages/MultiChannelColors.jl",
devbranch="main",
)
18 changes: 18 additions & 0 deletions docs/src/api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# API reference

## Type hierarchy and construction

```@docs
AbstractMultiChannelColor
MultiChannelColor
ColorMixture
GreenMagenta
MagentaGreen
```

## Fluorophores

```@docs
fluorophore_rgb
@fluorophore_rgb_str
```
26 changes: 26 additions & 0 deletions docs/src/faq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# FAQ

## Why are the RGB colors encoded in the `ColorMixture` *type*? Why not a value field?

In many places, JuliaImages assumes that you can convert from one color space to another purely from knowing the type you want to convert to. This would not be possible if the RGB colors were encoded as a second field of the color.

If you consider an entire image `Array{<:ColorMixture}` (the preferred default representation for code in JuliaImages), it becomes clear that storing the RGB colors as a value field would also require additional memory for each pixel.

## I wrote some code and got lousy performance. How can I fix it?

To achieve good performance, in some cases the RGB *values* must be aggressively constant-propagated, a feature available only on Julia 1.7 and higher. So if you're experiencing this problem on Julia 1.6, try a newer version.

If you're using fluorophore colors with `fluorophore_rgb`, where possible make sure you're using the compile-time constant syntax `fluorophore_rgb"EGFP"` rather than the runtime syntax `fluorophore_rgb["EGFP"]`.

When you can't get good performance otherwise, your best option is to use a [function barrier](https://docs.julialang.org/en/v1/manual/performance-tips/#kernel-functions):

```julia
ctemplate = ColorMixture((rgb1, rgb2))

@noinline function make_image_and_do_something(ctemplate, sz)
img = [ctemplate(rand(), rand()) for i = 1:sz[1], j = 1:sz[2]]
...
end
```

In this case `ctemplate` encodes the type, and code in `make_image_and_do_something` will be inferrable even if the type of the created `ctemplate` is not inferrable in its creation scope.
185 changes: 179 additions & 6 deletions docs/src/index.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,187 @@
```@meta
CurrentModule = FluorophoreColors
CurrentModule = MultiChannelColors
```

# FluorophoreColors
# MultiChannelColors

Documentation for [FluorophoreColors](https://github.com/JuliaImages/FluorophoreColors.jl).
[MultiChannelColors](https://github.com/JuliaImages/MultiChannelColors.jl) aims to support "unconventional colors," such as might arise in applications like multichannel fluorescence microscopy and hyperspectral imaging. Consistent with the philosophy of the [JuliaImages ecosystem](https://juliaimages.org/latest/), this package allows you to bundle together the different color channels into a "color object," and many color objects can be stored in an array. Having each entry of the array represent a complete pixel or voxel makes it much easier to write generic code supporting a wide range of image types.

```@index
## Installation

Install the package with `add MultiChannelColors` from the `pkg>` prompt, which you access by typing `]` from the `julia>` prompt. See the [Pkg documentation](https://pkgdocs.julialang.org/v1/getting-started/) for more information.

## Usage

Use the package interactively or in code with

```jldoctest demo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a confirmation: is the jldoctest deliberately used here rather than the @repl block?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, though there are many ways to solve the same problem. Basically I want doctests to make sure the code keeps working, and I don't want to have to use setup=:(using MultiChannelColors) on each doctest. So I created a named doctest, demo, and am loading the package in the first instance.

julia> using MultiChannelColors
```

In addition to giving access to specific types defined below, this will import the namespaces of [FixedPointNumbers](https://github.com/JuliaMath/FixedPointNumbers.jl) (which harmonizes the interpretation of "integer" and "floating-point" pixel-encodings) and [ColorTypes](https://github.com/JuliaGraphics/ColorTypes.jl) (which defines core color types and low-level manipulation). It will also define arithmetic for colors such as RGB (see [ColorVectorSpace](https://github.com/JuliaGraphics/ColorVectorSpace.jl)).

The color types in this package support two fundamental categories of operations:

- arithmetic operations such as `+` and `-` and multiplying or dividing by a scalar. You can also scale each color channel independently with `⊙` (obtained with `\odot<tab>`) or its synonym `hadamard`, e.g., `g ⊙ c` where `c` is a color object defined in this package and `g` is a tuple of real numbers (the "gains").
- extracting the independent channel intensities as a tuple with `Tuple(c)`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL 😆


When creating `c`, you have two choices which primarily affect visualization:

- to use ["bare" colors](@ref index_multichannelcolor) that store the multichannel data but lack any default conversion to other color spaces. This might be most appropriate if you have more than 3 channels, for which there may be many different ways to visualize the data they encode.
- to use [colors with built-in conversion to RGB](@ref index_colormixture), making them work automatically in standard visualization tools. This may be most appropriate when you have 3 or fewer channels.

Both options will be discussed below. See the [JuliaImages documentation on visualization](https://juliaimages.org/latest/install/#sec_visualization) for more information about tools for viewing images.

### ["Bare" colors: `MultiChannelColor`](@id index_multichannelcolor)

A `MultiChannelColor` object is essentially a glorified tuple, one that can be recognized as a [`Colorant`](https://github.com/JuliaGraphics/ColorTypes.jl#the-type-hierarchy-and-abstract-types) but with comparatively few automatic behaviors. For example, if you're working with [Landsat 8](https://en.wikipedia.org/wiki/Landsat_8) data with
[11 wavelength bands](https://landsat.gsfc.nasa.gov/satellites/landsat-8/landsat-8-bands/), one might create a pixel this way:

```jldoctest demo
julia> c = MultiChannelColor{N4f12}(0.2, 0.1, 0.2, 0.2, 0.25, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2)
(0.2N4f12₀₁, 0.1001N4f12₀₂, 0.2N4f12₀₃, 0.2N4f12₀₄, 0.2501N4f12₀₅, 0.2N4f12₀₆, 0.2N4f12₀₇, 0.2N4f12₀₈, 0.2N4f12₀₉, 0.2N4f12₁₀, 0.2N4f12₁₁)
```

See the [FixedPointNumbers](https://github.com/JuliaMath/FixedPointNumbers.jl) package for information about the 16-bit data type `N4f12` (Landsat 8 quantizes with 12 bits).

The usual way to visualize such an object is to define a custom function that converts such colors to more conventional colors (`RGB` or `Gray`). For example, we might compute the [Enhanced Vegetation Index](https://www.usgs.gov/landsat-missions/landsat-enhanced-vegetation-index)
and render positive values in green and negative values in magenta:

```jldoctest demo
julia> function evi(c::MultiChannelColor{T,11}) where T<:FixedPoint
# Valid for Landsat 8 with 11 spectral bands
b = Tuple(c) # extract the bands
evi = 2.5f0 * (b[5] - b[4]) / (b[5] + 6*b[4] - 7.5f0*b[2] + eps(T))
return evi > 0 ? RGB(0, evi, 0) : RGB(-evi, 0, -evi)
end;

julia> evi(c)
RGB{Float32}(0.0f0,0.17894554f0,0.0f0)
```

```@autodocs
Modules = [FluorophoreColors]
If `img` is a whole image of such pixels, `evi.(img)` converts the entire array to RGB. For large data, you might prefer to use the [MappedArrays package](https://github.com/JuliaArrays/MappedArrays.jl) to do such conversions "lazily" (on an as-needed basis) to avoid exhausting computer memory:

```julia
julia> using MappedArrays

julia> imgrgb = mappedarray(evi, img);
```

### [RGB-convertible colors: `ColorMixture`](@id index_colormixture)

`ColorMixture` objects are like `MultiChannelColor` objects except they have a built-in conversion to RGB. Each channel gets assigned a specific RGB color, say `rgbⱼ` for the `j`th channel, along with an intensity `iⱼ`.
`rgbⱼ` is a feature of the *type* (shared by all objects of the same type) whereas `iⱼ` is a property of *objects*.

`ColorMixture` objects are converted to RGB with intensity-weighting,

``
c_{rgb} = \sum_j i_j \mathrm{rgb}_j
``

Depending on the the `rgbⱼ` and `iⱼ`, values may exceed the 0-to-1 colorscale of RGBs.
Conversion to `RGB{Float32}` may be safer than `RGB{T}` where `T` is limited to 0-to-1.
It is also faster, as the result does not have to be checked for whether it exceeds the bounds of the type.
(To prevent overflow, all internal operations are performed using floating-point intermediates even if you want a `FixedPoint` output.)

!!! note
While `ColorMixture` objects can be converted to RGB, they are *not* AbstractRGB
colors: `red(c)`, `green(c)`, and `blue(c)` are not defined for `c::ColorMixture`, and low-level utilities
like `mapc` operate on the raw channel intensities rather than the RGB values.


There are several ways you can create these colors. An easy approach is to define the type through a "template" object:

```jldoctest demo
julia> ctemplate = ColorMixture{Float32}((RGB(0,1,0), RGB(1,0,0)))
(0.0₁, 0.0₂)
```

`ctemplate` is an all-zeros `ColorMixture` object, but can be used to construct arbitrary `c` with specified intensities:

```jldoctest demo
julia> typeof(ctemplate)
ColorMixture{Float32, 2, (RGB{N0f8}(0.0,1.0,0.0), RGB{N0f8}(1.0,0.0,0.0))}

julia> c = ctemplate(0.2, 0.4)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat!

(0.2₁, 0.4₂)

julia> Tuple(c)
(0.2f0, 0.4f0)
```

You can also create them with a single call `ColorMixture(rgbs, intensities)`:

```jldoctest demo
julia> c = ColorMixture{Float32}((RGB(0,1,0), RGB(1,0,0)), (0.2, 0.4))
(0.2₁, 0.4₂)
```

or even by explicit type construction:

```jldoctest demo
julia> ColorMixture{Float32, 2, (RGB{N0f8}(0.0,1.0,0.0), RGB{N0f8}(1.0,0.0,0.0))}(0.2, 0.4)
(0.2₁, 0.4₂)
```

!!! tip
All but the last form require [constant propagation](https://en.wikipedia.org/wiki/Constant_folding) for inferrability.
Julia 1.7 and higher can use "aggressive" constant propagation to solve inference problems that may reduce performance on Julia 1.6.

### Importing external data

When objects are not created by code but instead loaded from an external source such as a file, you have several avenues for creating arrays of multichannel color objects. There are two particularly common cases:

1. If the imported data are an array `A` of size `(nc, m, n)`, where `nc` is the number of color channels (i.e., color is the fastest dimension), then use `reinterpret(reshape, C, A)` where `C` is the color type you want to use (e.g., `MultiChannelColor{T,nc}` or `ColorMixture{T,nc,rgbs}`). For instance, Landsat 8 data might look something like this:

```julia
A = rand(0x0000:0x0fff, 11, 100, 100);
img = reinterpret(reshape, MultiChannelColor{N4f12,11}, A);
```

2. If the imported data have the color channel last, or use separate arrays for each channel, use the [StructArrays package](https://github.com/JuliaArrays/StructArrays.jl). For example:

```julia
A = rand(0x0000:0x0fff, 100, 100, 11);
img = StructArray{MultiChannelColor{N4f12,11}}(A; dims=3)
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we provide a better tool in ImageCore in the spirit of JuliaImages/ImageCore.jl#179, then this story might be changed.
Might be worth leaving a note about future improvement.


I was thinking of something like:

module ImageLayouts
const CHW = DenseImageLayout{:CHW}()
const CWH = DenseImageLayout{:CWH}()
const HWC = DenseImageLayout{:HWC}()
end

so that users can use a unified interface:

using ImageCore # which exports the ImageLayouts module

colorview(ImageLayouts.CHW, img)

we might end up with a new function, though -- pretty much like the strel concept I introduced in ImageMorphology.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I really like the idea of these custom layouts!


It is possible that simpler syntaxes will be developed in future releases.

## Additional features

### Fluorophores

This package also exports a lookup table for common [fluorophores](https://en.wikipedia.org/wiki/Fluorophore). If desired, these can be used as the `rgbⱼ` values for `ColorMixture` channels. For example:

```jldoctest demo
julia> channels = (fluorophore_rgb["EGFP"], fluorophore_rgb["tdTomato"])
(RGB{N0f8}(0.0,0.925,0.365), RGB{N0f8}(1.0,0.859,0.0))

julia> ctemplate = ColorMixture{N0f16}(channels)
(0.0N0f16₁, 0.0N0f16₂)
```

If you'll be hard-coding the name of the fluorophore, consider using a slightly different syntax:

```jldoctest demo
julia> channels = (fluorophore_rgb"EGFP", fluorophore_rgb"tdTomato")
Copy link
Member

@johnnychen94 johnnychen94 May 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fluorophore"EGFP" be easier to use? I'm not sure. -- just that I have never seen underscores used in string macros.

Is there non-RGB fluorophore?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No non-RGB fluorophore currently. I guess I would think that fluorophore"EGFP" might return an entire object that is a complete characterization of the fluorophore, e.g., full spectra, cross section, quantum yield, and so on (this list only scratches the surface). In contrast, fluorophore_rgb makes it clear that what you're getting is an RGB value that represents the fluorophore.

(RGB{N0f8}(0.0,0.925,0.365), RGB{N0f8}(1.0,0.859,0.0))
```

Note the absence of `[]` brackets around the fluorophore names. This form creates types inferrably, but the fluorophore name must be a literal string constant.

The RGB values are computed from the peak emission wavelength of each fluorophore; note, however, that the perceptual appearance is often more red-shifted due to the asymmetric shape of emission spectra.

### Green/magenta coloration

For good separability in two-color imaging, the `GreenMagenta{T}` and `MagentaGreen{T}` types are convenient:

```jldoctest demo
julia> c = GreenMagenta{N0f8}(0.2, 0.4)
(0.2N0f8₁, 0.4N0f8₂)

julia> convert(RGB, c)
RGB{N0f8}(0.4,0.2,0.4)
```

Green and magenta are distinguishable even by individuals with common forms of color blindness, and is thus a good default for two-color imaging.
Loading