Implement piecewise function #689

Kolaru · 2024-11-28T21:41:29Z

Implement piecewise function, as described in #655 (and handling decorations).

For example, the abs function would be defined as

myabs = Piecewise(
    interval(-Inf, 0) => x -> -x,
    interval(0, Inf) => identity ;
    continuous = true
)

It also introduce the Constant constructor, as a workaround for Returns from Base: the later can not convert its output to interval when the input is an interval, which causes problems.

For example:

julia> f = Returns(1.0)
Returns{Float64}(1.0)

julia> f(22.2)
1.0

julia> f(interval(22.2))
1.0

julia> g = Constant(1.0)
Constant{Float64}(1.0)

julia> g(22.2)
1.0

julia> g(interval(22.2))
[1.0, 1.0]_com

It can be used for example to define a step function as

step = Piecewise(
    interval(-Inf, 0) => Constant(0),
    interval(0, Inf) => Constant(1)
)

OlivierHnt · 2024-11-28T23:02:57Z

Oh nice, looks great!

One thing is that there should maybe be a check to prevent overlapping domains. Eg the two following functions do not behave the same, and are essentially ill-defined

julia> foo = Piecewise(
                  interval(-Inf, 0) => x -> -x,
                  interval(-1, 1) => x -> 2x,
                  interval(0, Inf) => x -> x ;
                  continuous = true
              )

julia> goo = Piecewise(
                  interval(-Inf, 0) => x -> -x,
                  interval(0, Inf) => x -> x,
                  interval(-1, 1) => x -> 2x ;
                  continuous = true
              )

Moreover, domain should probably not be a convex hull, maybe a vector of interval (or a union type, related to #424 in some sense)? I also wonder if the resulting interval need to always have the trv decoration. Eg

Julia> julia> f = Piecewise(
                         interval(-1, 0) => identity, 
                         interval(1, 2) => identity);

julia> domain(f)
[-1.0, 2.0]_trv

Julia> julia> g = Piecewise(
                         interval(-1, 0) => identity, 
                         interval(0, 1) => identity);

julia> domain(g) # should it have the decoration `trv`?
[-1.0, 2.0]_trv

One question on the implementation. Currently you can only indicate if a Piecewise function is continuous or not. This may be mitigated by creating nested Piecewise functions... but what would you think about

struct Piecewise2{N,T<:NTuple{N}}
    pieces :: T 
    continuous :: NTuple{N,Bool}
end

Piecewise2(pieces::NTuple{N}, continuous::Bool) where {N} = Piecewise2(pieces, ntuple(_ -> continuous, Val(N)))

This way you do not hit a def decoration when evaluating the piecewise function at continuous points.

dpsanders · 2024-11-28T23:57:09Z

src/piecewise.jl

+
+function (piecewise::Piecewise)(x::Real)
+    for (region, f) in piecewise.pieces
+        in_interval(x, region) && return f(x)


This doesn't seem completely right: you need to return the union of f(x) over all the pieces that x overlaps with, I think.

e.g. the line

return reduce(hull, f(intersect_interval(X, region)) for (region, f) ∈ piecewise.pieces)

in my original implementation in the issue you limked to.

This part is the implementation for non-interval real only, so I think it is fine, as it would be a bit awkward to return an interval for non-interval input.

As @OlivierHnt mentioned, this implementation allows ill-defined functions, but I don't think the answer is the interval-ification of everything.

Kolaru · 2024-11-30T01:32:24Z

One thing is that there should maybe be a check to prevent overlapping domains. Eg the two following functions do not behave the same, and are essentially ill-defined

Yes, that reasonable.

To do that properly, we would need open and closed intervals (otherwise the boundary points are always included, and thus points on the boundary of pieces are always duplicated).

I am wondering if it would be worth to pull intervals from Intervals.jl to do this.

Moreover, domain should probably not be a convex hull, maybe a vector of interval (or a union type, related to #424 in some sense)? I also wonder if the resulting interval need to always have the trv decoration. Eg

Actually yes, here you would have a bug if there is a hole in the middle of the pieces.

For the decoration, I don't know. Those intervals, really have not much in common with the arithmetic intervals...

This way you do not hit a def decoration when evaluating the piecewise function at continuous points.

I thought about this a bit, I am not strongly against implementing it. It is just a bit awkward that the order of the pieces matter in the input.

OlivierHnt · 2024-11-30T10:48:10Z

Ah I did not notice the behaviour when evaluating at the boundary point. So it is also order dependent

julia> foo = Piecewise(
           interval(-Inf, 0) => x -> -x,
           interval(0, Inf) => x -> 3 ;
           continuous = false
       );

julia> foo(0)
0

julia> goo = Piecewise(
           interval(0, Inf) => x -> 3,
           interval(-Inf, 0) => x -> -x;
           continuous = false
       );

julia> goo(0)
3

My first impulse would be not to add a dependency on Intervals.jl. One idea could be to introduce yet an other option to specify the behaviour. Eg Piecewise(...; continuous = (false, boundary_behaviour)), where boundary_behaviour::Symbol could be :left, :right, :average (maybe the latter ought to be the default).

Actually yes, here you would have a bug if there is a hole in the middle of the pieces.

Right, so for f = Piecewise(interval(0, 1) => x -> 1, interval(10, 11) => x -> 2), I would expect domain(f) to return something in the vein of (interval(0, 1, trv), interval(10, 11, trv)).
EDIT: Ah but this is already implemented as subdomains. Now this feels even more confusing to me. Also, the error is different whether the input is an Interval or a Real

julia> Piecewise(interval(0, 1) => x -> 1, interval(10, 11) => x -> 2)(interval(5.))
ERROR: ArgumentError: reducing over an empty collection is not allowed; consider supplying `init` to the reducer

Maybe I am overlooking something, but I think that the behaviour for not being in the domain should be the same regardless whether you are in the convex hull or not. Eg compare the above error with

julia> Piecewise(interval(0, 1) => x -> 1, interval(10, 11) => x -> 2)(interval(-2))
∅_trv

For the decoration, I don't know. Those intervals, really have not much in common with the arithmetic intervals...

Yes you are right, maybe trv is appropriate after all.

I thought about this a bit, I am not strongly against implementing it. It is just a bit awkward that the order of the pieces matter in the input.

Indeed.

OlivierHnt · 2024-11-30T18:52:54Z

Mhm I gave it a bit more thoughts. Here is an attempt (order independent)

struct Piecewise{T<:Tuple}
    pieces :: T
    # The symbol indicates whether or not the boudnary is included in the interval:
    # Symbol(11) means that the interval contains both extremities
    # Symbol(10) means that the interval contains only at the left extremity
    # Symbol(01) means that the interval contains only at the right extremity
    # Symbol(00) means that the interval does not contain either extremity
    # TODO: need to normalize `pieces`
    # TODO: missing `continuous` (to determine the decoration `def`, or `com`)
end

Piecewise(pieces...) = Piecewise(pieces)

_subdomains(piecewise::Piecewise) = first.(piecewise.pieces)

function _intersect_subdomain(x::Interval, region::Tuple{Interval,Symbol})
    y = IntervalArithmetic.setdecoration(intersect_interval(x, region[1]), decoration(x))
    if isempty_interval(y)
        return y
    elseif region[2] === Symbol(11)
        return y
    elseif region[2] === Symbol(01)
        isthin(y, inf(region[1])) && return emptyinterval(y)
        return y
    elseif region[2] === Symbol(10)
        isthin(y, sup(region[1])) && return emptyinterval(y)
        return y
    else # region[2] === Symbol(00)
        isthin(y, inf(region[1])) | isthin(y, sup(region[1])) && return emptyinterval(y)
        return y
    end
end

function (piecewise::Piecewise)(x::Interval)
    subdoms = _subdomains(piecewise)
    vals = _intersect_subdomain.(x, subdoms)
    all(v -> isempty_interval(v[1]), vals) && return throw(DomainError("piecewise function was called with $x which is outside of its domain $subdoms"))
    return _evaluate(piecewise.pieces, vals, nothing)
end

function _evaluate(p, x, r) # maybe not type stable
    isempty_interval(x[1][1]) && return _evaluate(Base.tail(p), Base.tail(x), r)
    fx = last(p[1])(x[1][1])
    return _evaluate(Base.tail(p), Base.tail(x), IntervalArithmetic.setdecoration(hull(fx, r), min(decoration(fx), decoration(r))))
end

function _evaluate(p, x, ::Nothing) # maybe not type stable
    isempty_interval(x[1][1]) && return _evaluate(Base.tail(p), Base.tail(x), nothing)
    fx = last(p[1])(x[1][1])
    return _evaluate(Base.tail(p), Base.tail(x), fx)
end

function _evaluate(p::Tuple{Pair}, x, r) # maybe not type stable
    isempty_interval(x[1][1]) && return r
    fx = last(p[1])(x[1][1])
    return IntervalArithmetic.setdecoration(hull(fx, r), min(decoration(fx), decoration(r)))
end

function _evaluate(p::Tuple{Pair}, x, ::Nothing) # maybe not type stable
    isempty_interval(x[1][1]) && return x[1][1]
    return last(p[1])(x[1][1])
end

then

foo = Piecewise((interval(-Inf, 0), Symbol(11)) => x -> -x,
                (interval( 0, Inf), Symbol(01)) => x -> interval(3));

foo(interval(-1, 0))

foo(interval(-1, 1)) # decoration is wrong, currently not implemented

foo(interval(0, 1))

foo(interval(0))

goo = Piecewise((interval(0, 1), Symbol(00)) => x -> interval(1),
                (interval(2, 3), Symbol(01)) => x -> interval(3));

goo(interval(0)) # now it throws

goo(interval(1, 2)) # now it throws

goo(interval(2, 3))

Kolaru · 2024-12-05T15:57:17Z

I will try to implement the ForwardDiff extension for this, it may give some directions as to how to to handle the boundary points.

Kolaru · 2025-01-03T18:26:12Z

In the current version (which still has a bug in the derivative computation), I am using Interval and IntervalSet from Intervals. It is very convenient, as they allow to specify inclusion of the boundary points and implement all required set operations.

To avoid confusion, I am aliasing Intervals.Interval to IntervalArithmetic.Domain (exported), so that a piecewise function can be defined as

julia>     slide = Piecewise(
                   Domain{Open, Closed}(-Inf, -1) => x -> -2x - 1,  # `false, true` means open on the left, closed on the right
                   Domain{Open, Closed}(-1, 0) => x -> x^2,
                   Domain{Open, Open}(0, Inf) => Constant(0) ;
                   continuity = [1, 1]
           )
Piecewise function with 3 pieces:
  (-Inf .. -1.0] -> var"#135#137"()
  (-1 .. 0] -> var"#136#138"()
  (0.0 .. Inf) -> Constant{Int64}(0)

The domains need to be explicitely disjoint and ordered, which is checked at construction.

OlivierHnt · 2025-02-06T15:16:55Z

One thing I would like to discuss is how necessary it is to add the dependancy on IntervalSet. Ideally I think we should always try to keep everything "in house", a fortiori for small features.

Kolaru · 2025-02-06T15:38:50Z

It is not striclty necessary, the only things I import are the following:

import Intervals
import Intervals: IntervalSet, less_than_disjoint, Open, Closed

const Domain = Intervals.Interval

I have no strong feeling about it, it must just be noted that in this case Intervals.jl is doing exactly what we need (dealing with open/closed intervals, and computing overlaps).

One advantage of pulling the code out is to avoid weird error showing something went wrong with Interval.Interval that are very confusing, considering that the user is not facing anything named like that.

OlivierHnt · 2025-02-06T15:50:49Z

One advantage of pulling the code out is to avoid weird error showing something went wrong with Interval.Interval that are very confusing, considering that the user is not facing anything named like that.

Indeed 😞

Maybe if we use what we have for interval we can get just what we need without too much effort,

struct Domain{T<:Symbol,S<:Symbol,R<:NumTypes}
    left :: T
    right :: S
    interval :: Interval{R}
    Domain{T,S,R}() = ...
end

Kolaru · 2025-02-06T15:58:06Z

I will try that :)

I think that it will probably be better to not wrap an Interval here, just because we don't want any of the complication that come with it, just two numbers, bound types and some set operations is enough. If more is needed we can always provide conversion function to our Intervals or to Intervals.Interval (as an extension).

src/intervals/interval_operations/set_operations.jl

src/piecewise.jl

OlivierHnt · 2025-02-08T10:54:44Z

src/piecewise.jl

+    return val1 < val2
+end
+
+Base.in(x::Real, domain::Domain) = rightof(x, lowerbound(domain)) && leftof(x, upperbound(domain))


Mhm maybe we should only export Domain to the user and not overload Base functions for it.

Why not ? They seem quite natural operation to perform.

I am not entirely sure 😄. My reasoning is based on maintaining consistency : it feels somewhat inconsistent that we have in_interval for BareInterval, while for Domain, we overload Base.in. This is maybe a "flaw" or something to be improved, I do not know....
However, for now, it seems more prudent to first implement the necessary internal functionalities and then extend them incrementally on 0.22.x (this way we also avoid having to deal with breaking changes).

A Domain is a set, and never a number, so we don't have the same ambiguity we have for an Interval. I can't see how it could go wrong (famous words to regrets, I know ^^).

(famous words to regrets, I know ^^)

😆

A Domain is a set, and never a number, so we don't have the same ambiguity we have for an Interval

I agree. Note that I was mentioning BareInterval (not Interval) which is not a subtype of Number as well.

OlivierHnt · 2025-02-08T11:00:19Z

With this new Domain type, I think this PR can also address #661.

Kolaru · 2025-02-09T18:18:39Z

With this new Domain type, I think this PR can also address #661.

For clarity and brevity, I would keep it for a follow up PR.

A good question though is: would we then define function with limited domains (e.g. log) as Piecewise functions? It would add a bit of overhead, but then the logic of checking the domain would be common across the package.

OlivierHnt · 2025-02-09T19:10:19Z

Interesting. The main question is probably "how bad" this overhead is.
This may warrant opening a dedicated issue to discuss it.

src/IntervalArithmetic.jl

src/piecewise.jl

src/intervals/interval_operations/set_operations.jl

OlivierHnt · 2025-02-09T19:57:02Z

src/piecewise.jl

+struct Piecewise
+    domain::Vector
+    fs
+    continuity::Vector{Int}
+    singularities::Vector
+end


You probably already thought about it, but I would have imagined using Tuple here instead of Vector. I think these are good below length 16, or 32?

But maybe an implementation like

struct Piecewise{M,N,T<:NTuple{M},S<:NTuple{M},R<:NTuple{N}} domains :: T functions :: S continuity_orders :: NTuple{N,Int} singularities :: R function Piecewise{M,N,T,S,R}(domains, functions, continuity_orders, singularities) where {M,N,T<:NTuple{M},S<:NTuple{M},R<:NTuple{N}} @assert M == N+1 # more checks... return new{M,N,T,S,R}(domains, functions, continuity_orders, singularities) end end

is messy, or can hurt the performance?

We can use tuples. It's just a bit messy because domains and functions can different types, so the type has whole tuple type as type parameters. But that's mostly a technicality.

Indeed, I let you decide what you think is best 🙂

Kolaru · 2025-02-09T22:17:47Z

I've addressed all you comments and added docstrings and doc , therefore (if the doc builds properly), this is ready for a final review and merging.

OlivierHnt · 2025-02-09T23:01:01Z

src/piecewise.jl

+end
+
+function Piecewise(pairs::Vararg{Pair} ; continuity = fill(-1, length(pairs) - 1))
+    pairs = collect(pairs)


No need to collect since the code now uses Tuple

OlivierHnt · 2025-02-09T23:03:57Z

src/piecewise.jl

+function Piecewise(
+        domains::Vector{<:Domain},
+        fs,
+        continuity::Vector{Int} = fill(-1, length(domains) - 1))


Maybe these should be:

function Piecewise( domains::NTuple{N,Domain}, fs, continuity::Tuple{Vararg{Int}} = ntuple(i -> -1, Val(N-1))) where {N} # .....

OlivierHnt · 2025-02-09T23:05:03Z

src/piecewise.jl

+    return Piecewise(Tuple(domains), Tuple(fs), Tuple(continuity), Tuple(singularities))
+end
+
+function Piecewise(pairs::Vararg{Pair} ; continuity = fill(-1, length(pairs) - 1))


Similarly, maybe

function Piecewise(pairs::Vararg{Pair,N} ; continuity = ntuple(i -> -1, Val(N-1))) where {N} #....

OlivierHnt · 2025-02-09T23:17:49Z

Thank you so much @Kolaru for taking the time to address this issue. This is a great new functionality for IntervalArithmetic! 🥳

Aside from my minor comments, the only remaining item for me is that I am not convinced we should overload Base.in, Base.isempty, etc. for Domain at the moment, since we do not do it for BareInterval.
I am not necessarily saying what we have going on is the best, but I believe we should be consistent. E.g., it's peculiar to have the following three behaviours

julia> 1 in bareinterval(1, 2) # no method defined
ERROR: MethodError: [...]

julia> 1 in interval(1, 2) # it defaults to `Base.in` for `Number`
false

julia> 1 in Domain(:closed, :closed, 1, 2) # this should throw the same error as `1 in bareinterval(1, 2)`
true

If we have some (internal?) functions -- say domain_in, domain_isempty, domain_intersect etc -- we can still make the corresponding Base functions aliases later on without a major version bump.

Kolaru · 2025-02-10T11:05:06Z

For now I have remove the overload of base function for Domain. I think it can wait for now anyway, because using piecewise functions should not require the user to manipulate the domains. We'll have to discuss it when addressing #661

I also just did a little final bit of refactoring.

OlivierHnt · 2025-02-10T11:11:13Z

LGTM 🎉

dpsanders reviewed Nov 28, 2024

View reviewed changes

OlivierHnt linked an issue Dec 12, 2024 that may be closed by this pull request

Piecewise functions #655

Closed

Kolaru added 8 commits February 6, 2025 16:08

Prototype

dc7bdf0

Add basic tests and Constant

b503f57

Add tests for and correct out-of-domain behavior

6743696

New strategy defining pieces

b934dad

Use Intervals for the domain of the pieces

6966ddd

Compat with forwarddiff

5057415

Add tests

72f08f1

Finish and fix tests

fe77b77

Add deps

7cec2d6

Kolaru force-pushed the branching branch from 3597c00 to 7cec2d6 Compare February 6, 2025 15:29

Kolaru added 3 commits February 6, 2025 18:25

Remove Intervals.jl dep (not tested, need to go)

aa069af

Fix everything (except the forwarddiff extension)

fde8b0a

Fix the derivative too

81492c7

OlivierHnt reviewed Feb 8, 2025

View reviewed changes

src/intervals/interval_operations/set_operations.jl Outdated Show resolved Hide resolved

OlivierHnt reviewed Feb 8, 2025

View reviewed changes

src/piecewise.jl Outdated Show resolved Hide resolved

OlivierHnt reviewed Feb 8, 2025

View reviewed changes

src/piecewise.jl Outdated Show resolved Hide resolved

OlivierHnt reviewed Feb 8, 2025

View reviewed changes

Fix interval_diff

51db09e

Check bounds type at construction

af32069