Releases: pedrokehl/caminho
v1.7.2
Bug fixes
Bug
The system had a bug which caused inconsistencies in the back-pressure system that eventually lead to stuck process.
This happened when there were many executions (.run()) for the same reused Caminho instance, so when some of the operators threw an error in the execution, the back-pressure system would not decrement the pending items for the items that has generated but haven't finished all the operators, therefore, when back-pressure system compared the maxItemsFlowing with the current number of items flowing, it was still accounting for items that were not in memory anymore.
Over time, the Caminho instance was allowing less and less items to be in memory, according to the number of items that were flowing during a failed execution, so at some point, the number of "ghost" items would reach maxItemsFlowing and stop processing completely.
Fix
The back-pressure system currently accounts all runs for a given Caminho instance, this gives a very good control of how many items can be in memory at the same time, making your process memory allocation very predictable, and even executions more efficient, the goal of this fix was to NOT change this behaviour, therefore, fixing this bug took more effort than simply changing the back-pressure to be per execution.
The fix consisted of generating a runId for each execution, so this ID allows to control the back-pressure isolated, and reset all the "pending items" for a given run whenever the flow for the given execution is ended, no matter if it had an error or not.
In the fix, the runId was intentionally not part of the ValueBag, this was done to avoid internal values of Caminho being available to the step functions, therefore, the OperatorApplierWithRunId was created.
v1.7.1
Bug fixes
errors in pipe() operator now won't cause a bad consistency in the total maxItemsFlowing controlling system for backpressure
This bug would cause the backpressure to be considering itemsFlowing more than it should, and if number of errors would reach the defined maxItemsFlowing, the Caminho run would get stuck.
Dependencies
- Bump braces from 3.0.2 to 3.0.3
v1.7.0
Features
Allow logging for filter() functions, along with forwarding of errors from it to the onStepFinished hook
Bug fixes
reduce() now won't cause a -1 effect on the Backpressure
This bug would cause the backpressure to be considering itemsFlowing as -1 after each run that got a reduce operation.
It got fixed and tests were written to avoid this ever happening again.
v1.6.0
Features
Call onStepFinished on error, and send the error as parameter to the function
It allows a better error tracking and logging
Breaking Changes
Remove the keepLastValues parameter of run()
The parameter was just bring complexity and confusion to the usage, so it's better removed.
Bug fixes
Avoid seed mutation on .reduce() even when reduceFn does mutation to "acc" param
RxJs doesn't create a structureClone from the seed parameter when start processing, while developers can implement a reduce function that mutates the "acc", so to safely avoid conflicts between different runs, we copy the seed for new runs