Where filter on Aff<A> question

[kirill-gerasimenko-da]

Hi @louthy!

When I run the following computation (given GetValue1 and GetValue2 both return Aff<A>) and val == null then Error.New("cancelled") is produced in where filter internally. If I wanted to run another computation after that first one conditionally (like if computation1 was cancelled then do next thing) then I need to match the result of computation1 and in the Fail: error => {...} case check if the message of the exception was 'cancelled' and then do the logic. Is my assumption correct? Or there is more elegant way of doing this?

var computation 1 = 
  from val1 in GetValue1()
  where val1 != null
  from val2 in GetValue2(val1)
  select val2

Thanks!

[louthy]

where is a more 'brutal' way of cancelling an expression, a better way is to use FailEff to construct a contextual error.

I have some documentation I've been working on over the last few days that deals with the issues you're raising. One thing that is potentially missing is anything around catching specific errors - I'll be writing that up soon. If you need that info, let me know and I'll try and get it written quicker.

[kirill-gerasimenko-da]

Thanks for your reply! I would really appreciate!

[kirill-gerasimenko-da]

I've read that document you've referenced - it's still a tricky part to my why failing the computation is needed if I just have some conditional logic and want to short-circuit. Would it make more sense to have additional state Cancelled or something similar?

[kirill-gerasimenko-da]

For now - this looks like a solution, but with it's own downsides. For example the code of the error might collide with some already defined one.

        [HttpGet("do2")]
        public async Task<IActionResult> Get2(CancellationToken ct)
        {
            var getIndustriesCall = _database.Invoke(new GetIndustriesQuery(), ct);

            var industriesEff = match(await getIndustriesCall.Run(),
                Succ: industries => industries.Count > 0
                    ? SuccessEff(industries)
                    : FailEff<Lst<Industry>>(Error.New(100, "No industries")),
                Fail: FailEff<Lst<Industry>>);

            var apiCall = industriesEff.Bind(xi => _apiClient.CallSomeApi(xi[0]));

            var keyResult = match(await apiCall.Run(),
                Succ: apiKey => "OK, got the key",
                Fail: error => error switch
                {
                    {Code: 100} => "100 error",
                    _ => "generic error"
                });

            return Content(keyResult);

[louthy]

You probably need what I’m going to be releasing in the next few days:

https://github.com/louthy/language-ext/blob/main/Samples/TestBed/AffTests.cs

[kirill-gerasimenko-da]

@louthy will this approach work if I don't have runtime and just using Aff/Eff ad-hoc as in my example?

[louthy]

Yep, it supports all Aff and Eff types

[louthy]

As a bit of a proof-of-concept, I took your Get2 call and broke it up into a runtime, a stand-alone static controller, and a new Get2 method. I'm using the runtime version of Aff, so there's more setup needed, but the end result is something that could be unit-tested.

It's not required though: the calls to SomeApi<RT>.CallSomeApi and Db<RT>.GetIndustries() could be the non-runtime version of Aff.

First, I declared a couple of interfaces for the IO:

    public interface DatabaseIO
    {
        ValueTask<Seq<Industry>> GetIndustries();
    }

    public interface SomeApiIO
    {
        ValueTask<string> CallSomeApi(Industry industry);
    }

These represent your database interaction and 'Some API' (which presumably has side-effects).

Then I implement some 'live' versions of these:

    public readonly struct LiveDatabase : DatabaseIO
    {
        public static readonly DatabaseIO Default = new LiveDatabase();
        
        public ValueTask<Seq<Industry>> GetIndustries() =>
            new ValueTask<Seq<Industry>>(Seq<Industry>());
    }

    public readonly struct LiveSomeApi : SomeApiIO
    {
        public static readonly SomeApiIO Default = new LiveSomeApi();

        public ValueTask<string> CallSomeApi(Industry industry) =>
            new ValueTask<string>("some key");
    }

Obviously just placeholders. You could of course make unit-test friendly versions of these.

Next, I define a couple of trait types that will be used with the runtime to give it capabilities:

    public interface HasDB<RT>
        where RT : struct, HasDB<RT>
    {
        Eff<RT, DatabaseIO> DbEff { get; }
    }
    
    public interface HasSomeApi<RT>
        where RT : struct, HasSomeApi<RT>
    {
        Eff<RT, SomeApiIO> SomeApiEff { get; }
    }

To make access to these behaviours nice and declarative, I create a couple of static helper classes:

    public static class Db<RT>
        where RT : struct,
            HasCancel<RT>,
            HasDB<RT>
    {
        public static Aff<RT, Seq<Industry>> GetIndustries() =>
            default(RT).DbEff.MapAsync(db => db.GetIndustries());
    }

    public static class SomeApi<RT>
        where RT : struct, 
            HasCancel<RT>, 
            HasSomeApi<RT>
    {
        public static Aff<RT, string> CallSomeApi(Industry industry) =>
            default(RT).SomeApiEff.MapAsync(rt => rt.CallSomeApi(industry));
    }

Now we need a runtime that supports these traits. This is a relatively simple one, there's more info in the wiki about creating your own runtimes:

    public readonly struct YourRuntime : 
        HasCancel<YourRuntime>,
        HasDB<YourRuntime>,
        HasSomeApi<YourRuntime>
    {
        public YourRuntime(CancellationTokenSource src) =>
            (CancellationTokenSource, CancellationToken) = (src, src.Token);
        
        public YourRuntime(CancellationToken token) =>
            (CancellationTokenSource, CancellationToken) = (new CancellationTokenSource(), token);
        
        public CancellationToken CancellationToken { get; }
        public CancellationTokenSource CancellationTokenSource { get; }

        public YourRuntime LocalCancel =>
            new YourRuntime(new CancellationTokenSource());

        public Eff<YourRuntime, DatabaseIO> DbEff =>
            SuccessEff(LiveDatabase.Default);

        public Eff<YourRuntime, SomeApiIO> SomeApiEff =>
            SuccessEff(LiveSomeApi.Default);
    }

All of the above you do once to describe your side-effecting operations.

Next, I defined some well-known error types, to keep them consistent:

    public static class Err
    {
        public static readonly Error NoIndustries = (100, "No industries");
        public static readonly Error SafeError = (0, "There was a problem");
    }

My thoughts about hooking up Aff and Eff to work with ASP.NET controllers is that it would be better to lift the pure logic out of the controller method and into a static class, so ASP.NET doesn't infect the code. It also makes the controllers unit-testable.

So, this is the core logic broken out into a stand-alone static (and pure) controller:

    public static class YourController<RT> 
        where RT : struct, 
            HasCancel<RT>,
            HasDB<RT>,
            HasSomeApi<RT>
    {
        public static Aff<RT, string> Get2() =>
            from industries in Db<RT>.GetIndustries()
            from _          in guard(industries.Count > 0, Err.NoIndustries)
            from key        in SomeApi<RT>.CallSomeApi(industries.Head)
            select key;
    }

You can see the new guards feature here. But, most importantly, we can see the intent of the author. It is very hard from your original example to see through the ceremony and see the intent.

Finally, we need to update the ASP.NET controller itself:

    public class YourController
    {
        [HttpGet("do2")]
        public async Task<IActionResult> Get2(CancellationToken ct)
        {
            var operation = YourController<YourRuntime>.Get2()
                          | @catch(Err.NoIndustries, Err.NoIndustries.Message)
                          | @catch(Err.SafeError.Message);

            return Content(await operation.Run(new YourRuntime(ct)));
        }
    }

This calls the static controller method to get the Aff and then composes it with @catch:

• The first @catch matches on the 100 error and flips it from an Error to a success string with just the error-message in it.
• The second @catch catches all other errors and flips them to a SafeError.Message string to stop information leakage outside of the server domain.

@catch documentation can be found here

Finally, we run the operation with the runtime. The beauty of using the runtime version of Aff is that you'll get the cancellation token passed through all of your operations by default.

Other benefits to breaking the pure controller code out into a static class is that it's easy to add retry and timeout semantics:

    var operation = YourController<YourRuntime>
                       .Get2()
                       .Retry(Schedule.Recurs(5) | Schedule.Fibonacci(10*milliseconds))
                  | @catch(Err.NoIndustries, Err.NoIndustries.Message)
                  | @catch(Err.SafeError.Message);

This will retry a maximum of five times, and each time back-off using the fibonnaci sequence (summing the previous two back-off values to get the new one).

Timeouts are just as easy:

    var operation = YourController<YourRuntime>
                       .Get2()
                       .Timeout(2*seconds)
                  | @catch(Err.NoIndustries, Err.NoIndustries.Message)
                  | @catch(Errors.TimedOut, "operation timed out")
                  | @catch(Err.SafeError.Message);

When using the runtime version of Aff the timeout will automatically cancel the running operation. The non-runtime version will timeout and return, but the original operation continues (due to the lack of a cancellation token).

Hopefully that's given some food for thought, and that the new @catch and guard features are useful. Again, there's no requirement to use the runtime version of Aff - but it does give a very good way to build unit-testable controllers, and has declarative benefits, even if it requires more initial setup.

[kirill-gerasimenko]

@louthy thanks for such great explanation! I really appreciate these details - I'll have to process all this information and do some testing :)

Just right now I have a question - do you usually create one runtime for a whole application? It would seem really heavy this way... Or for different submodules - different runtimes? How do you usually organize it?

[louthy]

@kirill-gerasimenko

Just right now I have a question - do you usually create one runtime for a whole application? It would seem really heavy this way

The choice is yours and depends on the complexity of your application. You can use localAff and localEff to run sub-expressions with a different runtime. However the child runtime needs to be created from the parent one (and the capabilities of the parent runtime), and so you'll need enough in the parent to generate the child (to keep the computation pure). You could (for example) use a parent runtime with a HasFile<RT> trait to load in some configuration data which can be used to generate a sub-runtime that has config info that's not available to the parent runtime.

You could have an application level runtime that defines access to all of the IO it will ever need and also embedded environment data (your application config). And then create one for maybe just your data-access layer that can only talk to the database and has a single configuration value which is your connection-string. However, that has very little benefit over just using the parent runtime --- unless you really wanted to lock down the capabilities of the programmers writing code in that domain so that they can never accidentally add the HasFile<RT> trait.

Personally, I think it's best to start with a single application level runtime, and then refine it later if you need. That's a minor refactoring job, because the functions that you write to use the Aff<RT, A> and Eff<RT, A> only discriminate by trait (like HasFile<RT> etc.), so they will continue to work as before without modification as long as your sub-runtime has those traits.

[kirill-gerasimenko-da]

Got it!

So let's say I've got single runtime for the application, then I don't understand how to work with cancellation tokens (of course these could be passed as a regular parameter). In the example you've done above - APSNET controller's action is getting CancellationToken on each invocation, and you create new runtime each time, and pass that CT there, and it makes sense, but if I get to do single runtime for the app (and create it on app start) - then it won't work. Is sub-runtimes are for that etc.? I can't get a clue unfortunately.

[louthy]

but if I get to do single runtime for the app (and create it on app start) - then it won't work. Is sub-runtimes are for that etc.? I can't get a clue unfortunately.

Fundamentally the Aff is a reader monad. It has an environment that is passed through, however you can modify that environment using localAff and localEff. Part of the environment is the cancellation-token, and so you can create a local cancellation context like so:

   Aff<RT, X> operation = ...;

   var localOperation = localAff<RT, RT, X>(static env => env.LocalCancel, operation);

Then when localOperation is run it will create a local cancellation token automatically. So, if you call cancel<RT>() during that operation then only it will end and not all of the other threads using the same base runtime.

This is obviously a common pattern, and so there's a built-in function called localCancel:

   var localOperation = localCancel(operation);

You can also create forked operations if you want something that's fire-and-forget (but also cancellable).