-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doc: Use inner/outer splitter, analogize to zip/product #773
base: main
Are you sure you want to change the base?
Conversation
I think the only issue with this description is that @djarecka was using "inner splitter" to refer to the split that occurs on a specific node and then combined on that node (i.e. not propagated to downstream nodes). So there may be places in the comments and variable names where it could get a little confusing |
I was thinking that we could do a search/replace for |
Actually looking through the use of inner splitter, it just seems to refer to a split of a node within a workflow. @djarecka is that right? |
Additional minor wording changes for clarity.
7a4fb5f
to
d962cc5
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #773 +/- ##
==========================================
+ Coverage 88.64% 88.66% +0.02%
==========================================
Files 85 85
Lines 17719 17719
Branches 1538 1538
==========================================
+ Hits 15707 15711 +4
+ Misses 1644 1640 -4
Partials 368 368 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
It's possible that I've misunderstood previous conversations, but I don't recall "inner splitter" ever referring to something internal to a node or a workflow, so much as an analogy to an inner-product / inner-join. I would generally expect most splits to be combined before the final node in a workflow, which is frequently the way it's done with iterables (and always with MapNodes) in Nipype 1, but that doesn't seem like a thing that really needs a particular name. If we do, I would probably just figure out how we want to describe nodes in terms of their modification of the splitting state; workflows are just nodes, and the terminology can be used in both cases. To throw out some terms:
Examples (apologies if this gets the new syntax wrong): # Splitting node, splits over x
a = Task1().split(x=xList, y=yList).combine('y')
# Combining node, combines over x
b = Task2(var1=a.out).split(var2=var2List).combine('var2', 'a.x')
# Reshaping node, combines over x, splits over z
c = Task3(var1=a.out).split(x=zList).combine('a.x') The idea is that what matters is the impact on the state space for downstream nodes, not the entire sequence of manipulations that only that node and the workflow writer ever see. Anyway, that's getting off-topic for this PR. @satra @djarecka Would be good to have your understanding of how "scalar"/"inner" have been used. |
perhaps semantics are at play here and inner and nested splits are being conflated. if i remember somewhat i believe that nested splits (i.e. any level of splits set inside a workflow), need to be combined before one steps out of a workflow. i cannot remember now how a workflow (as a cacheable task object) deals with splits inside it (whether inner/outer), and especially if the workflow also has splits set on it's inputs. there were certain circumstances under which an output state could not be easily generated/tracked, and we forced that combines needed to happen. i think it was this nested condition. |
@djarecka would you be happy with these semantics if we did a search replace on "inner" and replaced it with "internal" to remove references to the old nomenclature? |
Additional minor wording changes for clarity.