Allow vectors for LinearMap storage

[dkarrasch]

This extends the maps fied of higher-order LinearMaps to be a vector, rather than only a tuple. It does not change default behavior. To construct such maps, one would need to call non-exported constructors directly ("expert use").

Closes #169. @oschulz

[codecov]

Codecov Report

Merging #171 (6ba6b94) into master (c804b06) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #171      +/-   ##
==========================================
+ Coverage   99.63%   99.64%   +0.01%     
==========================================
  Files          14       14              
  Lines        1089     1130      +41     
==========================================
+ Hits         1085     1126      +41     
  Misses          4        4              

Impacted Files	Coverage Δ
src/conversion.jl	100.00% <ø> (ø)
src/LinearMaps.jl	100.00% <100.00%> (ø)
src/blockmap.jl	99.05% <100.00%> (+<0.01%)	⬆️
src/composition.jl	100.00% <100.00%> (ø)
src/kronecker.jl	98.47% <100.00%> (+0.03%)	⬆️
src/linearcombination.jl	100.00% <100.00%> (ø)
src/show.jl	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c804b06...6ba6b94. Read the comment docs.

[oschulz]

Thanks a lot for this, @dkarrasch! Would you consider adding a specialization

Base.sum(maps::LinearMapTupleOrVector) = LinearCombination(...)

to this PR? It would be very helpful for our current use case in MGVI.jl.

[dkarrasch]

That already works via the +(::LinearMap, ::LinearMap) fallback (except for complete type inference, because if the vector has length 1, then you get a LinearMap, if it has length >1, then you get a LinearCombination). I'll add that as a test case. Somewhere in the tests, we already have a corresponding prod test.

[oschulz]

That already works via the +(::LinearMap, ::LinearMap) fallback

It does, and we're using it, but it results in a tuple-backed LinearCombination which results in a high compile-time cost if the number of elements in the sum is high. In our case, it's also variable, so many different LinearCombination types get created.

Specializing sum would result in an sum of an array of linear maps being turned into an array-backed LinearCombination. That would be very helpful to our use case (and potentially other too, I would hope).

[dkarrasch]

Now I got it. Then how about making sum return a tuple-backed LinearCombination for LinearMapTuple via the fallback and a vector-backed LinearCombination for LinearMapVector? Note that for vectors the constructor will not be fully inferred: the promoted eltype is not inferable.

[oschulz]

Then how about making sum return a tuple-backed LinearCombination for LinearMapTuple

Sounds good!

Actually, instead of specializing sum it might be better (more generic) to specialize

Base.mapreduce(::typeof(identity), ::typeof(Base.add_sum), ::AbstractVector{<:LinearMap}) = LinearCombination(...)

Note that for vectors the constructor will not be fully inferred: the promoted eltype is not inferable.

I'm not quite sure I got that - doesn't the vector Ms just get wrapped in the LinearCombination, effectively?

[oschulz]

the promoted eltype is not inferable.
I'm not quite sure I got that

Ah, sorry, it's about T. True, in the general case it's not inferable. But I think a very typical use case will be a sum over linear maps with the same element type. We could make it inferable/type-stable for that case at least. And operations on arrays with mixed element types aren't inferable anyway.

[dkarrasch]

So, do you think you'll need an overload of mean as well? Currently it works on vectors of linear maps, but doesn't respect the vector backend.

[oschulz]

So, do you think you'll need an overload of mean as well?

Having mean would be great!

[oschulz]

Saw your commits regarding mean - thanks a lot!

[oschulz]

Again, thanks a lot for this @dkarrasch ! Will this be v3.5.2 or v v3.6.0?

[dkarrasch]

You're very welcome! This is a new feature, so this will be v3.6. If you can share some showcase/benchmark for this feature, please leave a link here. I'd be curious to see what difference this makes in real-world usage.

[oschulz]

We can definitely run some comparison tests with MGVI.jl.

CC @apmypb, @jknollm, @vindex10

[cvsvensson]

To construct such maps, one would need to call non-exported constructors directly ("expert use").

These maps can also be constructed using Base.prod, which leads to an indexing problem if the number of maps is one.

using LinearMaps
m = LinearMap(cumsum,3)
typeof(m*m) #Tuple type
typeof(prod([m,m])) #Vector type
typeof(prod([m])) #Vector type, and leads to an indexing problem:
prod([m])*ones(3) #ERROR: BoundsError: attempt to access 1-element Vector{LinearMaps.FunctionMap{Float64, typeof(cumsum), Nothing}} at index [2]

The error originates from here:

LinearMaps.jl/src/composition.jl

Lines 207 to 211 in f8eaf98

	function _compositemul!(y::AbstractVecOrMat,
	A::CompositeMap,
	x::AbstractVecOrMat,
	source = similar(y, (size(A.maps[1],1), size(x)[2:end]...)),
	dest = similar(y, (size(A.maps[2],1), size(x)[2:end]...)))