Refactored Error type + Eff and Aff applicative functors

[louthy]

There have been a number of calls on the Issues page for a ValidationAsync monad, which although it's a reasonable request (and I'll get to it at some point I'm sure), when I look at the example requests, it seems mostly the requestors want a smarter error handling story in general (especially for the collection of multiple errors).

The error-type that I'm building most of the modern functionality around (in Fin, Aff, and Eff for example) is the struct type: Error. It has been designed to handle both exceptional and expected errors. But the story around multiple errors was poor. Also, it wasn't possible to carry additional information with the Error, it was a closed-type other than ability to wrap up an Exception - so any additional data payloads was cumbersome and ugly.

Extending the struct type to be more featureful was asking for trouble, as it was already getting pretty messy.

Error refactor

So, I've bitten the bullet and refactored Error into an abstract record type.

Error sub-types

There are a few built-in sub-types:

• Exceptional - An unexpected error
• Expected - An expected error
• ManyErrors - Many errors (possibly zero)

These are the key base-types that indicate the 'flavour' of the error. For example, a 'user not found' error isn't
something exceptional, it's something we expect to happen. An OutOfMemoryException however, is
exceptional - it should never happen, and we should treat it as such.

Most of the time we want sensible handling of expected errors, and bail out completely for something exceptional. We also want to protect ourselves from information leakage. Leaking exceptional errors via public APIs is a sure-fire way to open up more information to hackers than you would like. The Error derived types all try to protect against this kind of leakage without losing the context of the type of error thrown.

When Exceptional is serialised, only the Message and Code component is serialised. There's no serialisation of the inner Exception or its stack-trace. It is also possible to construct an Exceptional message with an alternative message:

    Error.New("There was a problem", exception);

That means if the Error gets serialised, we only get a "There was a problem" and an error-code.

Deserialisation obviously means we can't recover the Exception, but the state of the Error will still be Exceptional - so it's possible to carry the severity of the error across domain boundaries without leaking too much information.

Error methods and properties

Essentially an error is either created from an Exception or it isn't. This allows for expected errors to be represented without throwing exceptions, but also it allows for more principled error handling. We can pattern-match on the
type, or use some of the built-in properties and methods to inspect the Error:

• IsExceptional - true for exceptional errors. For ManyErrors this is true if any of the errors are exceptional.
• IsExpected - true for non-exceptional/expected errors. For ManyErrors this is true if all of the errors are expected.
• Is<E>(E exception) - true if the Error is exceptional and any of the the internal Exception values are of type E.
• Is(Error error) - true if the Error matches the one provided. i.e. error.Is(Errors.TimedOut).
• IsEmpty - true if there are no errors in a ManyErrors
• Count - 1 for most errors, or n for the number of errors in a ManyErrors
• Head() - To get the first error
• Tail() - To get the tail of multiple errors

You may wonder why ManyErrors could be empty. That allows for Errors.None - which works a little like Option.None. We're saying: "The operation failed, but we have no information on why; it just did".

Error construction

The Error type can be constructed as before, with the various overloaded Error.New(...) calls.

For example, this is an expected error:

    Error.New("This error was expected")

When expected errors are used with codes then equality and matching is done via the code only:

    Error.New(404, "Page not found");

And this is an exceptional error:

    try
    {
    }
    catch(Exception e)
    {
        // This wraps up the exceptional error
        return Error.New(e);
    }

Finally, you can collect many errors:

   Error.Many(Error.New("error one"), Error.New("error two"));

Or more simply:

    Error.New("error one") + Error.New("error two")

Error types with additional data

You can extend the set of error types (perhaps for passing through extra data) by creating a new record that inherits Exceptional or Expected:

public record BespokeError(bool MyData) : Expected("Something bespoke", 100, None); 

By default the properties of the new error-type won't be serialised. So, if you want to pass a payload over the wire, add the [property: DataMember] attribute to each member:

public record BespokeError([property: DataMember] bool MyData) : Expected("Something bespoke", 100, None); 

Using this technique it's trivial to create new error-types when additional data needs to be moved around, but also there's a ton of built-in functionality for the most common use-cases.

Error breaking changes

• Because Error isn't a struct any more, default(Error) will now result in null. In practice this shouldn't affect anyone.
• BottomException is now in LanguageExt.Common

Error documentation

There's also a big improvement on the API documentation for the Error types

Aff and Eff applicative functors

Now that Error can handle multiple errors, we can implement applicative behaviours for Aff and Eff. If you think of monads enforcing sequential operations (and therefore can only continue if each operation succeeds - leading to only one error report if it fails), then applicative-functors are the opposite in that they can run independently.

This is what's used for the Validation monads, to allow multiple operations to be evaluated, and then all of the errors collected.

By adding Apply to Aff and Eff, we can now do the same kind of validation-logic both synchronously and asynchronously.

Contrived example

First let's create a simple asynchronous effect that delays for a period of time:

    static Aff<Unit> delay(int milliseconds) =>
        Aff(async () =>
        {
            await Task.Delay(milliseconds);
            return unit;
        });

Now we'll combine that so we get an effect that parses a string into an int, and adds a delay of 1000 milliseconds (the delay is to simulate calling some external IO).
:

    static Aff<int> parse(string str) =>
        from x in parseInt(str).ToAff(Error.New("parse error: expected int"))
        from _ in delay(1000)
        select x;

Notice how we're converting the Option<int> to an Aff, and providing an error value to use if the Option is None

Next we'll use the applicative behaviour of the Aff to run two operations in parallel. When they complete the values will be applied to the function that has been lifted by SuccessAff.

    static Aff<int> add(string sx, string sy) =>
        SuccessAff((int x, int y) => x + y) 
            .Apply(parse(sx), parse(sy));

To measure what we're doing, let's add a simple function called report. All it does is run an Aff, measures how long it takes, and prints the results to the screen:

    static async Task report<A>(Aff<A> ma)
    {
        var sw = Stopwatch.StartNew();
        var r = await ma.Run();
        sw.Stop();
        Console.WriteLine($"Result: {r} in {sw.ElapsedMilliseconds}ms");
    }

Finally, we can run it:

    await report(add("100", "200"));
    await report(add("zzz", "yyy"));

The output for the two operations is this:

Result: Succ(300) in 1032ms
Result: Fail([parse error: expected int, parse error: expected int]) in 13ms

Notice how the first one (which succeeds) takes 1032ms - i.e. the two parse operations ran in parallel. And on the second one, we get both of the errors returned. The reason that one finished so quickly is because the delay was after the parseInt call, so we exited immediately.

Of course, it would be possible to do this:

   from x in parse(sx)
   from y in parse(sy)
   select x + y;

Which is more elegant. But the success path would take 2000ms, and the failure path would only report the first error.

Hopefully that gives some insight into the power of applicatives (even if they're a bit ugly in C#!)

Beta

This will be in beta for a little while, as the changes to the Error type are not trivial.

---

This discussion was created from the release Refactored `Error` type + `Eff` and `Aff` applicative functors.