Address distributed non-ordered indexing #914
Assignees
Labels
bug
Something isn't working
enhancement
New feature or request
❗
High priority, urgent
indexing
MPI
Anything related to MPI communication
redistribution
Related to distributed tensors
stale
Milestone
Comments
ClaudiaComito
added
❗
4 tasks
|
Branch bugs/914-Address_distributed_non-ordered_indexing created! |
|
This issue is stale because it has been open for 60 days with no activity. |
|
This issue is stale because it has been open for 60 days with no activity. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Related
All kind of stuff depends on distributed non-ordered indexing. Here a sample of issues/PRs where the problem has come up in various forms:
#607 #760 #903 #703 #824 #902 #177 #621 #749 #271 #857
Feature functionality
We want to be able to index a distributed DNDarray with a distributed, non-ordered key, and return the correct, stable result. Examples below. An implementation of this functionality via Alltoallv is available in
ht.sort(), needs to be generalized.>>> a = ht.arange(50, split=0) >>> a DNDarray([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49], dtype=ht.int32, device=cpu:0, split=0) >>> b = ht.random.randint(0,50,(20,), dtype=ht.int64, split=0) >>> b DNDarray([46, 5, 44, 8, 14, 10, 30, 15, 34, 30, 41, 44, 28, 26, 11, 20, 16, 7, 9, 8], dtype=ht.int64, device=cpu:0, split=0) >>> c = a[b] >>> c DNDarray([46, 5, 44, 8, 14, 10, 30, 15, 34, 30, 41, 44, 28, 26, 11, 20, 16, 7, 9, 8], dtype=ht.int32, device=cpu:0, split=0)In the current implementation,
c = a[b]returns a distributed DNDarray populated by whichever subset of the key is process-local.On 2 processes:
On 3 processes:
The text was updated successfully, but these errors were encountered: