Skip to content

Add keyword indexing + constructor #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 6, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ IterTools = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
RangeArrays = "b3c3ace0-ae52-54e7-9d0b-2c1406fd6b9d"

[compat]
IntervalSets = "0.1, 0.2, 0.3"
IntervalSets = "0.1, 0.2, 0.3, 0.4"
IterTools = "1"
RangeArrays = "0.3"
julia = "1"
Expand Down
69 changes: 41 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,43 @@
# AxisArrays
# AxisArrays.jl

[![Build Status](https://travis-ci.org/JuliaArrays/AxisArrays.jl.svg?branch=master)](https://travis-ci.org/JuliaArrays/AxisArrays.jl) [![Coverage Status](https://coveralls.io/repos/github/JuliaArrays/AxisArrays.jl/badge.svg?branch=master)](https://coveralls.io/github/JuliaArrays/AxisArrays.jl?branch=master)

This package for the Julia language provides an array type (the `AxisArray`) that knows about its dimension names and axis values.
This allows for indexing with the axis name without incurring any runtime overhead.
AxisArrays can also be indexed by the values of their axes, allowing column names or interval selections.
This allows for indexing by name without incurring any runtime overhead.
This permits one to implement algorithms that are oblivious to the storage order of the underlying arrays.
AxisArrays can also be indexed by the values along their axes, allowing column names or interval selections.

In contrast to similar approaches in [Images.jl](https://github.com/timholy/Images.jl) and [NamedArrays.jl](https://github.com/davidavdav/NamedArrays), this allows for type-stable selection of dimensions and compile-time axis lookup. It is also better suited for regularly sampled axes, like samples over time.

Collaboration is welcome! This is still a work-in-progress. See [the roadmap](https://github.com/JuliaArrays/AxisArrays.jl/issues/7) for the project's current direction.

### Notice regarding `axes`
### Note about `Axis{}` and keywords

An `AxisArray` stores an object of type `Axis{:name}` for each dimension,
containing both the name (a `Symbol`) and the "axis values" (an `AbstractVector`).
These types are what made compile-time lookup possible.
Instead of providing them explicitly, it is now possible to use keyword arguments
for both construction and indexing:

```julia
V = AxisArray(rand(10); row='a':'j') # AxisArray(rand(10), Axis{:row}('a':'j'))
V[row='c'] == V[Axis{:row}('c')] == V[row=3] == V[3]
```

### Note about `axes()` and `indices()`

Since Julia version 0.7, the name `axes` is exported by default from `Base`
with a meaning (and behavior) that is distinct from how AxisArrays has been
using it. Since you cannot simultaneously be `using` the same name from the two
different modules, Julia will issue a warning, and it'll error if you try to
use `axes` without qualification:
The function `AxisArrays.axes` returns the tuple of such `Axis` objects.
Since Julia version 0.7, `Base.axes(V) == (1:10,)` gives instead the range of possible
ordinary integer indices. (This was called `Base.indices`.) Since both names are exported,
this collision results in a warning if you try to use `axes` without qualification:

```julia
julia> axes([])
julia> axes([1,2])
WARNING: both AxisArrays and Base export "axes"; uses of it in module Main must be qualified
ERROR: UndefVarError: axes not defined
```

Packages that are upgrading to support 0.7+ and use AxisArrays should follow
this upgrade path:
Packages that are upgrading to support Julia 0.7+ should:

* Replace all uses of the `axes` function with the fully-qualified `AxisArrays.axes`
* Replace all uses of the deprecated `indices` function with the un-qualified `axes`
Expand All @@ -38,14 +50,13 @@ path to whatever the new name will be.
## Example of currently-implemented behavior:

```julia
julia> using Pkg; Pkg.add("AxisArrays")
julia> using AxisArrays, Unitful
julia> import Unitful: s, ms, µs
julia> using Random: MersenneTwister
julia> using Pkg; pkg"add AxisArrays Unitful"
julia> using AxisArrays, Unitful, Random

julia> rng = MersenneTwister(123) # Seed a random number generator for repeatable examples
julia> fs = 40000 # Generate a 40kHz noisy signal, with spike-like stuff added for testing
julia> y = randn(rng, 60*fs+1)*3
julia> fs = 40000; # Generate a 40kHz noisy signal, with spike-like stuff added for testing
julia> import Unitful: s, ms, µs
julia> rng = Random.MersenneTwister(123); # Seed a random number generator for repeatable examples
julia> y = randn(rng, 60*fs+1)*3;
julia> for spk = (sin.(0.8:0.2:8.6) .* [0:0.01:.1; .15:.1:.95; 1:-.05:.05] .* 50,
sin.(0.8:0.4:8.6) .* [0:0.02:.1; .15:.1:1; 1:-.2:.1] .* 50)
i = rand(rng, round(Int,.001fs):1fs)
Expand All @@ -55,7 +66,7 @@ julia> for spk = (sin.(0.8:0.2:8.6) .* [0:0.01:.1; .15:.1:.95; 1:-.05:.05] .* 50
end
end

julia> A = AxisArray([y 2y], Axis{:time}(0s:1s/fs:60s), Axis{:chan}([:c1, :c2]))
julia> A = AxisArray(hcat(y, 2 .* y); time = (0s:1s/fs:60s), chan = ([:c1, :c2]))
2-dimensional AxisArray{Float64,2,...} with axes:
:time, 0.0 s:2.5e-5 s:60.0 s
:chan, Symbol[:c1, :c2]
Expand Down Expand Up @@ -87,14 +98,14 @@ information to enable all sorts of fancy behaviors. For example, we can specify
indices in *any* order, just so long as we annotate them with the axis name:

```julia
julia> A[Axis{:time}(4)]
julia> A[time=4] # or A[Axis{:time}(4)]
1-dimensional AxisArray{Float64,1,...} with axes:
:chan, Symbol[:c1, :c2]
And data, a 2-element Array{Float64,1}:
1.37825
2.75649

julia> A[Axis{:chan}(:c2), Axis{:time}(1:5)]
julia> A[chan = :c2, time = 1:5] # or A[Axis{:chan}(:c2), Axis{:time}(1:5)]
1-dimensional AxisArray{Float64,1,...} with axes:
:time, 0.0 s:2.5e-5 s:0.0001 s
And data, a 5-element Array{Float64,1}:
Expand Down Expand Up @@ -127,9 +138,13 @@ julia> AxisArrays.axes(ans, 1)
AxisArrays.Axis{:time,StepRangeLen{Quantity{Float64, Dimensions:{𝐓}, Units:{s}},Base.TwicePrecision{Quantity{Float64, Dimensions:{𝐓}, Units:{s}}},Base.TwicePrecision{Quantity{Float64, Dimensions:{𝐓}, Units:{s}}}}}(5.0e-5 s:2.5e-5 s:0.0002 s)
```

You can also index by a single value on an axis using `atvalue`. This will drop
a dimension. Indexing with an `Interval` type retains dimensions, even
when the ends of the interval are equal:
You can also index by a single value using `atvalue(t)`.
This function is not needed for categorical axes like `:chan` here,
as `:c1` is a `Symbol` which can't be confused with an integer index.

Using `atvalue()` will drop a dimension (like using a single integer).
Indexing with an `Interval(lo, hi)` type retains dimensions, even
when the ends of the interval are equal (like using a range `1:1`):

```julia
julia> A[atvalue(2.5e-5s), :c1]
Expand Down Expand Up @@ -220,8 +235,6 @@ across the columns.

## Indexing

### Indexing axes

Two main types of Axes supported by default include:

* Categorical axis -- These are vectors of labels, normally symbols or
Expand All @@ -238,7 +251,7 @@ headers.

```julia
B = AxisArray(reshape(1:15, 5, 3), .1:.1:0.5, [:a, :b, :c])
B[Axis{:row}(0.2..0.4)] # restrict the AxisArray along the time axis
B[row = (0.2..0.4)] # restrict the AxisArray along the time axis
B[0.0..0.3, [:a, :c]] # select an interval and two of the columns
```

Expand Down
5 changes: 5 additions & 0 deletions src/core.jl
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,11 @@ function AxisArray(A::AbstractArray{T,N}, names::NTuple{N,Symbol}, steps::NTuple
AxisArray(A, axs...)
end

# Alternative constructor, takes names as keywords:
AxisArray(A; kw...) = AxisArray(A, nt_to_axes(kw.data)...)
@generated nt_to_axes(nt::NamedTuple) =
Expr(:tuple, (:(Axis{$(QuoteNode(n))}(getfield(nt, $(QuoteNode(n))))) for n in nt.names)...)

AxisArray(A::AxisArray) = A
AxisArray(A::AxisArray, ax::Vararg{Axis, N}) where N =
AxisArray(A.data, ax..., last(Base.IteratorsMD.split(axes(A), Val(N)))...)
Expand Down
18 changes: 17 additions & 1 deletion src/indexing.jl
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ Base.IndexStyle(::Type{AxisArray{T,N,D,Ax}}) where {T,N,D,Ax} = IndexStyle(D)
# Cartesian iteration
Base.eachindex(A::AxisArray) = eachindex(A.data)

# Avoid an ambiguity -- [email protected] takes .. from EllipsisNotation,
# which defines A[..] for any AbstractArray, like this:
Base.getindex(A::AxisArray, ::typeof(..)) = A

"""
reaxis(A::AxisArray, I...)

Expand Down Expand Up @@ -129,7 +133,6 @@ end
@propagate_inbounds getindex_converted(A, idxs...) = A.data[idxs...]
@propagate_inbounds setindex!_converted(A, v, idxs...) = (A.data[idxs...] = v)


# First is indexing by named axis. We simply sort the axes and re-dispatch.
# When indexing by named axis the shapes of omitted dimensions are preserved
# TODO: should we handle multidimensional Axis indexes? It could be interpreted
Expand All @@ -155,6 +158,19 @@ function Base.reshape(A::AxisArray, ::Val{N}) where N
AxisArray(reshape(A.data, Val(N)), Base.front(axN))
end

# Keyword indexing, reconstructs the Axis{}() objects
@propagate_inbounds Base.view(A::AxisArray; kw...) =
view(A, kw_to_axes(parent(A), kw.data)...)
@propagate_inbounds Base.getindex(A::AxisArray; kw...) =
getindex(A, kw_to_axes(parent(A), kw.data)...)
@propagate_inbounds Base.setindex!(A::AxisArray, val; kw...) =
setindex!(A, val, kw_to_axes(parent(A), kw.data)...)

function kw_to_axes(A::AbstractArray, nt::NamedTuple)
length(nt) == 0 && throw(BoundsError(A, ())) # Trivial case A[] lands here
nt_to_axes(nt)
end

### Indexing along values of the axes ###

# Default axes indexing throws an error
Expand Down
5 changes: 5 additions & 0 deletions test/core.jl
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,11 @@ B = AxisArray([1 4; 2 5; 3 6], (:x, :y), (0.2, 100))
B = AxisArray([1 4; 2 5; 3 6], (:x, :y), (0.2, 100), (-3,14))
@test axisnames(B) == (:x, :y)
@test axisvalues(B) == (-3:0.2:-2.6, 14:100:114)
# Keyword constructor
C = AxisArray([1 4; 2 5; 3 6], x=10:10:30, y=[:a, :b])
@test axisnames(C) == (:x, :y)
@test axisvalues(C) == (10:10:30, [:a, :b])
@test @inferred(AxisArray(parent(C), x=1:3, y=1:2)) isa AxisArray

@test AxisArrays.HasAxes(A) == AxisArrays.HasAxes{true}()
@test AxisArrays.HasAxes([1]) == AxisArrays.HasAxes{false}()
Expand Down
9 changes: 9 additions & 0 deletions test/indexing.jl
Original file line number Diff line number Diff line change
Expand Up @@ -302,3 +302,12 @@ a = [2, 3, 7]
@test a[idx] ≈ 6.2
aa = AxisArray(a, :x)
@test aa[idx] ≈ 6.2

# Keyword indexing
A = AxisArray([1 2; 3 4], Axis{:x}(10:10:20), Axis{:y}(["c", "d"]))
@test @inferred(A[x=1, y=1]) == 1
@test @inferred(A[x=1]) == [1, 2]
@test axisnames(A[x=1]) == (:y,)
@test @inferred(view(A, x=1)) == [1,2]
@test parent(view(A, x=1)) isa SubArray
@test @inferred(A[x=atvalue(20), y=atvalue("d")]) == 4
6 changes: 6 additions & 0 deletions test/readme.jl
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,14 @@ for spk = (sin.(0.8:0.2:8.6) .* [0:0.01:.1; .15:.1:.95; 1:-.05:.05] .* 50,
end

A = AxisArray([y 2y], Axis{:time}(0s:1s/fs:60s), Axis{:chan}([:c1, :c2]))
A = AxisArray(hcat(y, 2 .* y); time = (0s:1s/fs:60s), chan = ([:c1, :c2]))

A[Axis{:time}(4)]
A[time=4]

A[Axis{:chan}(:c2), Axis{:time}(1:5)]
A[chan = :c2, time = 1:5]

ax = A[40µs .. 220µs, :c1]
AxisArrays.axes(ax, 1)
A[atindex(-90µs .. 90µs, 5), :c2]
Expand Down
2 changes: 1 addition & 1 deletion test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ using Random
import IterTools

@testset "AxisArrays" begin
VERSION >= v"1.0.0-" && @test isempty(detect_ambiguities(AxisArrays, Base, Core))
VERSION >= v"1.1" && @test isempty(detect_ambiguities(AxisArrays, Base, Core))

@testset "Core" begin
include("core.jl")
Expand Down