tables without deltas don't replenish downstream tables

[jhellerstein]

(courtesy @ilikerps)
Given a rule like

derived <+ base

and a delete from derived, we'd expect that base would "replenish" the deleted values from derived. This doesn't currently happen. See code below:

require 'rubygems'
require 'bud'

class Test
  include Bud
  state do
    table :myTable, [:str]
    table :other, [:str]
  end

  bloom do
    other <+ myTable
  end
end

t = Test.new
t.myTable <+ [["Hello!"]]
t.tick
t.tick
puts t.other.entries # prints ["Hello!"]

t.other <- [["Hello!"]]
t.tick
t.tick
puts t.myTable.entries # prints ["Hello!"]
puts t.other.entries   # prints []

[aarondav]

And additionally, even with a delta, only the delta is added to the downstream table.

[neilconway]

This might be related to #292 (which, unfortunately, seems like it will take some work to fix properly). I'll take a look...

[neilconway]

Ugh, there are at least 3 different errors in the rescan/invalidate logic interacting here; similar to the problems underlying #292, but at least some of the problems are distinct.

1. When a table receives a deletion, we mark it as invalidated; we we also want to (a) mark the table's scanner, if any, for rescan (b) mark any downstream nodes as both invalidated and needing rescan (c) mark any upstream nodes as needing rescan. Note that (b) is because downstream state should be blown away to prep for this table's rescan, whereas (c) is because we want to refill the table that received the deletion (e.g., because there might be another way to derive the deleted tuple). The problem is that the set of other elements to rescan/invalidate is associated with the table's scanner -- but only tables that appear on the RHS of rules have scanners (other would not have a scanner above). So the fix here is probably to move the set of to-rescan/to-invalidate elements to the collection itself.
2. The code for computing the set of elements to rescan/invalidation upon deletion was just wrong; for each scanner s, the previous coding considered "what elements would need rescan/invalidation were s to need rescan?" This is the wrong hypothetical; the situation is that s has been invalidated, not that it needs rescan (e.g., if a scanner on a table is to be rescanned, we don't also need to rescan upstream nodes; whereas if the table is to be invalidated, we do).
3. Next, the code neglected to consider the need to rescan nodes connected via <+ rules; it only did <= rules. Or at least, I suspect this is fishy; need to fix the first two issues first, which are problems regardless.

[aarondav]

Thanks for the fixes! Will this cause stdio <~ to behave more intuitively as well? It seemed to have the same problem.

[neilconway]

Nope; that issue is #292 (#297 is also related), which needs fixing separately. That's on my TODO list; hopefully I'll get a chance to fix it today or tomorrow.