Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an example Python script that shows how to generate I/O size histogram from nsys report #535

Draft
wants to merge 28 commits into
base: branch-25.02
Choose a base branch
from
Draft

Add an example Python script that shows how to generate I/O size histogram from nsys report #535

wants to merge 28 commits into from

Conversation

kingcrimsontianyu
Copy link
Contributor

No description provided.

@kingcrimsontianyu kingcrimsontianyu added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Nov 5, 2024
@kingcrimsontianyu kingcrimsontianyu self-assigned this Nov 5, 2024
@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented Dec 12, 2024

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@kingcrimsontianyu kingcrimsontianyu changed the base branch from branch-24.12 to branch-25.02 December 12, 2024 13:55
raydouglass and others added 22 commits January 23, 2025 15:03
@raydouglass
DOC v25.04 Updates [skip ci]
2d5fb60
@kingcrimsontianyu
Apply east const using clang-format. Fix header issues reported by cl…
ff6f841 
…ang-tidy (rapidsai#594)

This small PR applies east const style using `clang-format`. This makes KvikIO consistent with cuDF in coding style. The following parameters were applied to `.clang-format` for auto-reformatting. The file `.clang-format` itself is not updated in this PR.
```
QualifierAlignment: Custom
QualifierOrder: [inline, static, type, const, volatile]
```
In addition, this PR fixes minor "missing header" issues reported by `clang-tidy`.

Authors:
  - Tianyu Liu (https://github.com/kingcrimsontianyu)

Approvers:
  - Vukasin Milovanovic (https://github.com/vuule)
  - Mads R. B. Kristensen (https://github.com/madsbk)

URL: rapidsai#594
@bdice
Merge branch-25.02 into branch-25.04
b7be217
@AyodeAwe
Merge pull request rapidsai#610 from bdice/branch-25.04-merge-25.02
0940856 
Forward-merge branch-25.02 to branch-25.04
@bdice
Migrate to NVKS for amd64 CI runners (rapidsai#605)
6b0836e 
This migrates amd64 CI jobs (PRs and nightlies) to use L4 GPUs from the NVKS cluster.

xref: rapidsai/build-infra#184

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Gil Forsyth (https://github.com/gforsyth)

URL: rapidsai#605
@jameslamb
disallow fallback to Make in Python builds (rapidsai#612)
3630b8c 
Contributes to rapidsai/build-planning#146

Proposes:

* setting `[tool.scikit-build].ninja.make-fallback = false`, so `scikit-build-core` will not silently fallback to using GNU Make if `ninja` is not available

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: rapidsai#612
@kingcrimsontianyu
Add RAII file wrappers to avoid resource leak (rapidsai#614)
255ed48 
Closes rapidsai#607

Authors:
  - Tianyu Liu (https://github.com/kingcrimsontianyu)

Approvers:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)
  - Mads R. B. Kristensen (https://github.com/madsbk)
  - Vukasin Milovanovic (https://github.com/vuule)

URL: rapidsai#614
@vyasr
Fix package versioning (rapidsai#616)
b001413 
The nightly package versions are currently not being correctly assigned.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)
  - James Lamb (https://github.com/jameslamb)

URL: rapidsai#616
@gforsyth
Add shellcheck to pre-commit and fix warnings (rapidsai#621)
099657c 
`shellcheck` is a fast, static analysis tool for shell scripts. It's good at
flagging up unused variables, unintentional glob expansions, and other potential
execution and security headaches that arise from the wonders of `bash` (and
other shlangs).

This PR adds a `pre-commit` hook to run `shellcheck` on all of the `sh-lang`
files in the `ci/` directory, and the changes requested by `shellcheck` to make
the existing files pass the check.

xref: rapidsai/build-planning#135

Authors:
  - Gil Forsyth (https://github.com/gforsyth)

Approvers:
  - James Lamb (https://github.com/jameslamb)
  - Mads R. B. Kristensen (https://github.com/madsbk)

URL: rapidsai#621
@gforsyth
Add build_type input field for test.yaml (rapidsai#620)
6b1a71f 
Exposes `build_type` as an input in `test.yaml` so that `test.yaml` can be
manually run against a specific branch/commit as needed.

The default value is still `nightly`, and without maintainer intervention, that
is what will run each night.

xref rapidsai/build-planning#147

Authors:
  - Gil Forsyth (https://github.com/gforsyth)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: rapidsai#620
@bdice
Use shared-workflows branch-25.04 (rapidsai#623)
5b57e49 
This completes the migration to NVKS runners now that all libraries have been tested and rapidsai/shared-workflows#273 has been merged.

xref: rapidsai/build-infra#184

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: rapidsai#623
@bdice
Merge branch-25.02 into branch-25.04
31f4af8
@msarahan
add telemetry (rapidsai#615)
7dcf216 
Enables telemetry during kvikio CI runs. This is done by parsing GitHub Actions run log metadata and should have no impact on build or test times.

xref rapidsai/build-infra#139

Authors:
  - Mike Sarahan (https://github.com/msarahan)
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: rapidsai#615
@bdice
Use rapids-generate-version for package versions (rapidsai#628)
5a825df 
This uses the `RAPIDS_PACKAGE_VERSION` values set in rapidsai#616. This ensures we have consistent nightly versions.

This PR is independent of rapidsai#622 (it is needed regardless of whether that PR is closed or merged).

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: rapidsai#628
@raydouglass
Merge pull request rapidsai#626 from bdice/branch-25.04-merge-25.02
c38038e 
Forward-merge branch-25.02 into branch-25.04
@GPUtester
Merge pull request rapidsai#633 from rapidsai/branch-25.02
f171d74 
Forward-merge branch-25.02 into branch-25.04
@KyleFromNVIDIA
Consolidate more Conda solves in CI (rapidsai#636)
ecb81c2 
Issue: rapidsai/build-planning#22

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: rapidsai#636
@kingcrimsontianyu
Adjust the way of determining FileHandle's compatibility mode for syn…
887679b 
…c and async I/O to improve code readability (rapidsai#608)

This PR improves the readability of compatibility mode handling.

The current way of determining FileHandle's compatibility mode is somewhat complicated and unintuitive. The data member `_compat_mode` accompanied by some utility functions more or less combines 3 different things into one:

- The initially requested compat mode (`ON`/`OFF`/`AUTO`)
- The capability of performing synchronous cuFile I/O (bool)
- The capability of performing asynchronous cufile I/O (bool)

The disadvantages include:
- `FileHandle::is_compat_mode_preferred()` always derives the preferred compat mode on the fly as opposed to getting an already determined value.
- `FileHandle::is_compat_mode_preferred_for_async(CompatMode)` is potentially throwing, which is asymmetric to `is_compat_mode_preferred()`. Also when the compat mode is `OFF`, it has to invoke `is_stream_api_available()` and `config_path()` on each pass instead of getting an already determined value.
- There is no way to retrieve what the original requested compat mode is.

These add to cognitive burden when rereading the source to introduce new features to FileHandle. This PR attempts to improve the logic by making it concise and crystal clear.

This PR also fixes a line number bug in error handling.

This PR is breaking in that the rarely used public functions to query the compat mode data in the `FileHandle` are removed. These data are instead queryable via the new `CompatModeManager` class.

Authors:
  - Tianyu Liu (https://github.com/kingcrimsontianyu)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)
  - Lawrence Mitchell (https://github.com/wence-)

URL: rapidsai#608
@kingcrimsontianyu
Enable tracing of thread pool tasks using NVTX (rapidsai#630)
096ac0f 
This PR implements the basic feature outlined in rapidsai#631. 
The two good-to-haves are currently blocked.

Authors:
  - Tianyu Liu (https://github.com/kingcrimsontianyu)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)
  - Lawrence Mitchell (https://github.com/wence-)

URL: rapidsai#630
@yiheng-wang-nv
Add tutorials to showcase GDS for nifti and dicom images (rapidsai#597)
77da587 
This PR addresses the tutorial contribution mentioned in rapidsai#580

Authors:
  - Yiheng Wang (https://github.com/yiheng-wang-nv)
  - Mads R. B. Kristensen (https://github.com/madsbk)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)

URL: rapidsai#597
@GPUtester
Merge pull request rapidsai#640 from rapidsai/branch-25.02
974a9d8 
Forward-merge branch-25.02 into branch-25.04
@KyleFromNVIDIA
Add verify-codeowners hook (rapidsai#611)
a9bc84c 
Issue: rapidsai/pre-commit-hooks#61

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)
  - James Lamb (https://github.com/jameslamb)

URL: rapidsai#611
kingcrimsontianyu and others added 6 commits February 21, 2025 07:33
@kingcrimsontianyu
Minor fixes on the file permission, host allocation flag, and worker …
08ab53b 
…thread initialization (rapidsai#637)

This PR makes the following minor fixes:
- Use the correct file permission flags corresponding to the `644` code.
- Use the correct flag for the `cuMemHostAlloc` call.
- For the thread pool, replace the `thread_local` call-once section (which may negatively affect performance; see rapidsai#630 (comment)) with more idiomatic worker thread initialization function.

Authors:
  - Tianyu Liu (https://github.com/kingcrimsontianyu)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)
  - Vukasin Milovanovic (https://github.com/vuule)

URL: rapidsai#637
@Intron7
add install docs (rapidsai#638)
fcd84e3 
This adds documentation on how to install KvikIO from PyPI

Authors:
  - Severin Dicks (https://github.com/Intron7)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)

URL: rapidsai#638
@kingcrimsontianyu
Add initial script
33e4ea6
@kingcrimsontianyu
Fix formatting issue
24c3bcd
@kingcrimsontianyu
Fix formatting issue
e01048d
@kingcrimsontianyu
Improve messages printed
3c29f6b
@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented Feb 21, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@kingcrimsontianyu kingcrimsontianyu

Labels
improvement Improves an existing functionality non-breaking Introduces a non-breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.