Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand distributed indexing, match numpy indexing scheme #938

Draft
wants to merge 150 commits into
base: main
Choose a base branch
from

Conversation

ClaudiaComito
Copy link
Contributor

@ClaudiaComito ClaudiaComito commented Mar 24, 2022

Description

DRAFT

Intro: main points from Numpy array indexing scheme

Heat: indexing massive, memory-distributed arrays

process-local indexing

distributed indexing

  • sorted key
  • non-sorted key

Memory footprint

Scaling behaviour

Issue/s resolved: #914 #918

Changes proposed:

Type of change

Memory requirements

Performance

Due Diligence

  • All split configurations tested
  • Multiple dtypes tested in relevant functions
  • Documentation updated (if needed)
  • Updated changelog.md under the title "Pending Additions"

Does this change modify the behaviour of other functions? If so, which?

yes / no

skip ci

Mystic-Slice and others added 10 commits April 8, 2022 10:56
…ays (#937)

* Create ci.yaml

* Update ci.yaml

* Update ci.yaml

* Create CITATION.cff

* Update CITATION.cff

* Update ci.yaml

different python and pytorch versions

* Update ci.yaml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Delete pre-commit.yml

* Update ci.yaml

* Update CITATION.cff

* Update tutorial.ipynb

delete example with different split axis

* Delete logo_heAT.pdf

Removal of old logo

* ht.nonzero() returns tuple of 1-D arrays instead of n-D arrays

* Updated documentation and Unit-tests

* replace x.larray with local_x

* Code fixes

* Fix return type of nonzero function and gout value

* Made sure DNDarray meta-data is available to the tuple members

* Transpose before if-branching + adjustments to accomodate it

* Fixed global shape assignment

* Updated changelog

Co-authored-by: mtar <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Coquelin <[email protected]>
Co-authored-by: Markus Goetz <[email protected]>
Co-authored-by: Claudia Comito <[email protected]>
@ClaudiaComito ClaudiaComito modified the milestones: 1.4.0, 1.5.0 Apr 12, 2024
Copy link
Contributor

This pull request is stale because it has been open for 60 days with no activity.

@github-actions github-actions bot added the stale label Jul 29, 2024
@ClaudiaComito ClaudiaComito modified the milestones: 1.5.0, 1.6 Aug 26, 2024
@github-actions github-actions bot removed the stale label Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Address distributed non-ordered indexing
3 participants