Pandas: ParticleContainer_*.to_df()
#220
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Copy all particles into a `pandas.DataFrame`. Supports local and MPI-gathered results.
ax3l
commented
Nov 5, 2023
| next_df[f"SoA_real_{idx}"] = array | ||
| for idx, array in enumerate(soa_np_int): | ||
| next_df[f"SoA_int_{idx}"] = array | ||
|
|
There was a problem hiding this comment.
todo: I think this might miss on the runtime attributes (they are also in SoA format).
This should go into SoA::to_numpy aka soa_to_numpy & soa_to_cupy,
There was a problem hiding this comment.
Requires to call:
AddRealComp- query current number of real components
- call
ResizeRuntimeRealCompagain with redundant info (again the comm and number of components)
but AMReX-Codes/amrex#3615 could simplify this.
1 task
This was referenced Nov 6, 2023
ax3l
added a commit
to ax3l/impactx
that referenced
this pull request
Nov 9, 2023
ax3l
added a commit
to ax3l/impactx
that referenced
this pull request
Nov 19, 2023
ax3l
added a commit
to ax3l/impactx
that referenced
this pull request
Nov 20, 2023
ax3l
added a commit
to ax3l/impactx
that referenced
this pull request
Nov 20, 2023
ax3l
added a commit
to ax3l/impactx
that referenced
this pull request
Nov 20, 2023
ax3l
added a commit
to ECP-WarpX/impactx
that referenced
this pull request
Nov 20, 2023
* Pandas: `ImpactXParticleContainer.to_df()` Copy all particles into a `pandas.DataFrame`. Supports local and MPI-gathered results. * AMReX & pyAMReX: latest development X-ref: AMReX-Codes/pyamrex#220 * DataFrame: Add Test
atmyers
pushed a commit
to AMReX-Codes/amrex
that referenced
this pull request
Mar 29, 2024
## Summary When adding new `Real`/`Int` runtime components, they could be made available without additional calls. The cost should be the same as calling it explicitly later, but clarifies the usage. Alternatively, we should add API contract details to the `AddRealComp`/`AddIntComp` doc strings to make sure people use it right. ## Additional background - AMReX-Codes/pyamrex#220 - AMReX-Codes/pyamrex#222 ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] include documentation in the code and/or rst files, if appropriate
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Copy all particles into a
pandas.DataFrame. Supports local and MPI-gathered results.Future Directions (Attribute Names in AMReX)
At the moment, we make up generic names for the AoS and SoA attributes when we convert them to numpy, cupy and pandas.
It would generally be nice if AMReX supported the functionality to name particle attributes. In WarpX, we added this as
NamedParticleContainer, but as a mainstream implementation we could avoid deriving yet another class type (because if we do, we will again have additional compile-time to bind that custom type in pyAMReX), and instead could just extend the particle classes with these runtime details.Future Directions (Write)
Once the AMReX SoA transition is complete in:
we might even be able to provide node-local, zero-copy write access in
PandasandcuDFparticle data frames... (follow-up PRs).