Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing multiplication by bmag in GKYL_PROJ_FUNC of gk_species_projection #305

Closed
wants to merge 837 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
837 commits
Select commit Hold shift + click to select a range
729a9b6
Merge branch 'gk-g0-app' into gk-g0-app-all-gather
JunoRavin Feb 8, 2024
0d866cc
Merge branch 'gk-g0-app' into gk-g0-app-bgk
JunoRavin Feb 8, 2024
2674955
Hooking in the all-gather to the GK app. Bug fix in null_comm to set …
JunoRavin Feb 8, 2024
8480c8e
Add an option to the projection routine to initialize lab moments (M0…
JunoRavin Feb 8, 2024
80a3e2f
Merge remote-tracking branch 'origin/gk-g0-app-bgk' into gk-g0-app-co…
JunoRavin Feb 8, 2024
c27a7bd
Small cleanup, but there is some problem with correct maxwellian for …
JunoRavin Feb 8, 2024
c0f1d4c
Merge branch 'gk-g0-app' into gk-g0-app_diffMod
manauref Feb 8, 2024
c8ddf5b
Adding a test from Akash to try to debug the 2x2v initialization. May…
JunoRavin Feb 8, 2024
716e7b4
Merge pull request #271 from ammarhakim/gk-g0-app_diffMod
akashukla Feb 8, 2024
1a3d50e
Update kernels with correct constant. Replace fit data with fits usin…
jRoeltgen Feb 8, 2024
aadcf68
Regression and unit test are valgrind clean
jRoeltgen Feb 8, 2024
203380c
Adding code to project Maxwellian for gk_neut species. Adding step rt…
tnbernard Feb 8, 2024
9bbdb92
Correcting integer division in d3d input file. Restoring initial res …
manauref Feb 9, 2024
3295327
Radiation unit and regression tests compile on gpus
jRoeltgen Feb 9, 2024
884245a
Adding the option to make the neutrals static so they don't evolve in…
JunoRavin Feb 9, 2024
8c6f25d
Fixing the memory management is the neutral species is static
JunoRavin Feb 9, 2024
4795b5e
Typing too fast and accidentally had a typo in the species name
JunoRavin Feb 9, 2024
ab6b4fc
Small typo/bug fix in find_neut_species. Change neutral reacts to rea…
akashukla Feb 9, 2024
e47601f
Merge branch 'gk-g0-app' into gk-g0-app-all-gather
JunoRavin Feb 9, 2024
be42fe5
small changes to ar reg test.
tnbernard Feb 9, 2024
6624727
Adding MPI to the 2x2v sheath test.
JunoRavin Feb 9, 2024
985d915
The deflated fem poisson needs to be careful about the local range it…
JunoRavin Feb 9, 2024
860e5f7
Add 2D unit test to ctest_dg_rad_gyrokinetic
jRoeltgen Feb 9, 2024
5ef23b9
Adding confRange_ext to inp structs in unit tests. Also fixing produc…
tnbernard Feb 9, 2024
644d61e
Changing conf grid range to extended for recomb unit test. It passes …
tnbernard Feb 9, 2024
3824006
Forgot to extend range of phase space grids.
tnbernard Feb 9, 2024
36a8b9f
Changes to unit tests to correct "ghost" arrays for conf grids.
tnbernard Feb 9, 2024
121910f
Adding print statements to identify cutoff density/temp for reactions.
tnbernard Feb 9, 2024
526ef69
Adding lines to calculate rhs for reactions (iz/recombination). Someh…
tnbernard Feb 9, 2024
fae2652
Adding rhs calculation for gk_neut species reactions.
tnbernard Feb 9, 2024
7d5fe87
Regression test runs on gpus
jRoeltgen Feb 10, 2024
8c3db2b
Forward progress (seemingly). Problem is that I need both the local r…
JunoRavin Feb 10, 2024
d45c85c
Fixing memory issues. As of right now the set_rhs method accesses gho…
JunoRavin Feb 10, 2024
44b5e07
Storing the sub-range that is the intersection of the local with the …
JunoRavin Feb 10, 2024
7666a51
I think it is more consistent to initialize the nodal range from the …
JunoRavin Feb 10, 2024
091c07a
Adding a 2x2v unit test for correct maxwellian updater
JunoRavin Feb 12, 2024
8640bcc
Getting rid of long lines in deflated_fem_poisson while I try to figu…
JunoRavin Feb 12, 2024
bb787fc
Tentatively seems to have fixed parallel simulations. The problem was…
JunoRavin Feb 12, 2024
d3568cf
In a number of places, neutrals were not using the neut_basis but the…
JunoRavin Feb 12, 2024
a5016bd
Reverting gk sheath 2x2v simulation back to original run time. Things…
JunoRavin Feb 12, 2024
59d7866
Changes to call correct function for gk_species_react when gk_neut sp…
tnbernard Feb 12, 2024
a56078e
Remove debug statements and extra comments
jRoeltgen Feb 12, 2024
54309dc
Merge branch 'gk-g0-app' into gk-g0-app-radiation
jRoeltgen Feb 12, 2024
f003bb2
Fixing component value in adas nodal array. Changes to recomb updater…
tnbernard Feb 12, 2024
9c2c0b7
Merge branch 'gk-g0-app-neut' of https://github.com/ammarhakim/gkylze…
tnbernard Feb 12, 2024
3503e3a
Adding simpler regression test for argon ions with static argon neutr…
tnbernard Feb 12, 2024
a7c5631
Decrease number of velocity cells in unit test
jRoeltgen Feb 12, 2024
f56a311
Initial code for doing geometry in parallel. Only mapc2p is implement…
akashukla Feb 12, 2024
6ab507a
Fixing recombination updater to define correct range to access adas d…
tnbernard Feb 12, 2024
d96a77a
Make tomakak geometry mpi friendly. Seems to produce right geometry i…
akashukla Feb 12, 2024
20700d9
correcting the size of diffD. Also using the correct metric factor no…
manauref Feb 12, 2024
0a372d7
Merge branch 'gk-g0-app' into gk-g0-app-all-gather-geo
akashukla Feb 13, 2024
d4e9729
Mirror geo now also works with mpi. Add a regression test for wham wh…
akashukla Feb 13, 2024
272bd1f
Realized a mistake from earlier where geo local ranges were not creat…
akashukla Feb 13, 2024
0c7fe3a
Merge pull request #273 from ammarhakim/gk-g0-app_bcApp
JunoRavin Feb 13, 2024
867549d
Merge remote-tracking branch 'origin/gk-g0-app' into gk-g0-app-radiation
JunoRavin Feb 13, 2024
bc023f1
Fixing GPU version of radiation and moment tests. I did not catch tha…
JunoRavin Feb 13, 2024
f0a3633
Fixing a Lua wrapper compiler error with the generalization to having…
JunoRavin Feb 13, 2024
4f1a6e3
Fixing GPU radiation test by making sure we are operating on GPU arra…
JunoRavin Feb 13, 2024
9df08f4
Merge branch 'gk-g0-app-radiation' of https://github.com/ammarhakim/g…
JunoRavin Feb 13, 2024
1828a43
Initial GPU fix was incorrect because gk_geom is always a device side…
JunoRavin Feb 13, 2024
5f8eedf
Typo in host-side array declaration
JunoRavin Feb 13, 2024
d87512a
More typos needing fixing. Can't redeclare variable
JunoRavin Feb 13, 2024
7cc6372
Merge pull request #274 from ammarhakim/gk-g0-app-radiation
JunoRavin Feb 13, 2024
ba75b1b
Remove the wham 1x mpi test since this is not supported
akashukla Feb 13, 2024
89626e0
Merge remote-tracking branch 'origin/gk-g0-app' into gk-g0-app-correct
JunoRavin Feb 13, 2024
3b35faf
Read and write geometry in serial using procedure suggested by Jimmy …
akashukla Feb 13, 2024
d38349e
Remove a chunk of comments i forgot about
akashukla Feb 13, 2024
561de35
Rearranged logic in app to have initialization on gpu always from fil…
akashukla Feb 14, 2024
6ca3092
Dividing jacobgeo out of m0 moments in neutral interactions. (Still n…
tnbernard Feb 14, 2024
ebe2c0e
Adding diagnostics for react (gk only). Not fully tested.
tnbernard Feb 14, 2024
0734c66
React and neut react moments now follow the logic of BGK. Jacobgeo is…
tnbernard Feb 14, 2024
150900a
Removing write statements from iz and recomb updaters. Changing logic…
tnbernard Feb 14, 2024
c437727
One small fix to dg_iz : cu method was being called with args in wron…
akashukla Feb 14, 2024
ee900a0
Merge main -> gk-g0-app
manauref Feb 14, 2024
2a88bf4
Adding react_neut diagnostics to write out iz and recomb reaction rates.
tnbernard Feb 14, 2024
c892652
Merge branch 'gk-g0-app-neut' of https://github.com/ammarhakim/gkylze…
tnbernard Feb 14, 2024
9af8a85
Correcting TEST_CHECK values and tolerances for unit tests.
tnbernard Feb 14, 2024
f028414
Fixing memory error in recomb cuda kernel. Releasing gkyl arrays in r…
tnbernard Feb 15, 2024
83ae71a
Small syntatical restructure copying mom_gyrokinetic, which does not …
JunoRavin Feb 15, 2024
61d0eef
Figured it out. This change to 1x2v and 2x2v did fix it. You always n…
JunoRavin Feb 15, 2024
f894bea
Slight amendment to 2x2v test to match 1x2v extents and resolution (s…
JunoRavin Feb 15, 2024
3ef74a8
Potential bug fix for correct maxwellian updater. The algorithm is de…
JunoRavin Feb 15, 2024
c61aa20
More refinement of unit test. Caught an issue in the 2x2v case. Also …
JunoRavin Feb 15, 2024
1c353b1
Small changes to get ionization/recomb to run on both gpu and cpu. In…
akashukla Feb 15, 2024
23f1e06
Do not set pointers to inizdata and recombdata, do a gkyl_array copy …
akashukla Feb 15, 2024
b84bef8
Adding 2x2v argon test before Tess merges in gk-g0-app
akashukla Feb 15, 2024
2aeef33
Merge branch 'gk-g0-app-neut' of https://github.com/ammarhakim/gkylze…
akashukla Feb 15, 2024
88db2e3
Geometry always reads from file on gpu. Tested on stellar - works
akashukla Feb 15, 2024
6cc872e
Fix some typos and also update unit tests to pass the global range ar…
akashukla Feb 15, 2024
152be62
Adding large factor to multiply reaction rate for checking unit test …
tnbernard Feb 15, 2024
ff62f7c
Removing write statements from unit test.
tnbernard Feb 15, 2024
02f2e53
Allocate separate host side react coef array when running on gpu to w…
akashukla Feb 15, 2024
ae6852f
Change all local range names in geo from range to local. Update comme…
akashukla Feb 16, 2024
1044156
Merge gk-g0-app -> gk-g0-app-all-gather. Unfortunately the way this w…
manauref Feb 16, 2024
618887d
Fixing compilation errors after merge. Now IWL case should run in par…
JunoRavin Feb 16, 2024
981d672
Slight cleanup of different field objects. Now 1x multi-species simul…
JunoRavin Feb 16, 2024
fe58208
Move augment local and augment grid methods to gk_geometry.c. Make a …
akashukla Feb 16, 2024
e6a3660
Geometry input struct holds all 3d grids,ranges,bases and also app->c…
akashukla Feb 16, 2024
c51546e
Remove a leftover cu file. Ready to merge
akashukla Feb 16, 2024
1d1730d
Implement nccl_all_gather. Unit tests pass.
manauref Feb 16, 2024
09bced7
Rename augment local and augment grid methods. Fix unit tests to matc…
akashukla Feb 16, 2024
032c25e
Rename the geo input struct
akashukla Feb 16, 2024
3bb8a0c
Remove apps header files from unit tests.
manauref Feb 16, 2024
261c87e
Deallocate the mpi_comm all_gather buffers in the right place. Implem…
manauref Feb 16, 2024
a0248b3
Fix tokamak regression tests that I did not maintain when geometry ch…
akashukla Feb 16, 2024
df4f95e
Merge branch 'gk-g0-app-all-gather-geo' of https://github.com/ammarha…
akashukla Feb 16, 2024
9ce16f1
Adding code to rescale to correct density after gk project on maxwell…
tnbernard Feb 16, 2024
0599831
Merge branch 'gk-g0-app-neut' of https://github.com/ammarhakim/gkylze…
tnbernard Feb 16, 2024
78f3ed9
Removing unnecessary comments.
tnbernard Feb 16, 2024
f3a844f
Fix all unit tests which use geom. Fix rmin and rmax error in unit t…
akashukla Feb 16, 2024
34fc3bf
Fix a regression tests and remove extra geo lines in radiation test. …
akashukla Feb 17, 2024
0693d50
some mkarr calls had args in wrong order
akashukla Feb 17, 2024
4d15bfa
Merge branch 'gk-g0-app-all-gather' into gk-g0-app-all-gather-geo
JunoRavin Feb 17, 2024
72f0721
Removing this regression test because we have a unit test that times …
JunoRavin Feb 17, 2024
b827c4f
Merge pull request #279 from ammarhakim/gk-g0-app-all-gather-geo
manauref Feb 17, 2024
1baf840
Caught two sets of memory leaks in initialization with allocated memo…
JunoRavin Feb 17, 2024
176744d
Merge branch 'gk-g0-app-all-gather' into gk-g0-app-all-gather_nccl
manauref Feb 17, 2024
433ba28
Changing elc recomb term so that m0_ion multiplies instead of m0_elc …
tnbernard Feb 19, 2024
94b4e42
Moving gkyl_set array for m0_ion from iz section to recomb section.
tnbernard Feb 19, 2024
1180793
Implement all_reduce_host method in gkyl_comm so that NCCL can do an …
manauref Feb 20, 2024
3c0fc52
Pass phibc to fem_parproj when cdim=1
manauref Feb 20, 2024
0508a21
Change indexing in some nodal_ops so they don't produce erroneous res…
manauref Feb 20, 2024
02256bf
Rename -imom file to _integrated_moms for clarity and to follow other…
manauref Feb 20, 2024
9c81311
Fiz fem_parproj BC in and add mpi/nccl to rt_gk_sheath_1x2v. Couldn't…
manauref Feb 20, 2024
bc8a1d3
Move initialization of global_sub_range in gk_field; it wasn't being …
manauref Feb 21, 2024
b07d9b5
Fix a typo where react and recomb cu kernel function signatures took …
Feb 22, 2024
69880bc
Rename all_gather -> allgather and all_reduce -> allreduce.
manauref Feb 22, 2024
b82630f
Add source moments and integrated moments diagnostics to app
akashukla Feb 22, 2024
609e4d5
Change name -imom to _integrated_moms
akashukla Feb 22, 2024
76d6fcf
Implement mpi_comm array_bcast method and its unit tests, which pass.
manauref Feb 22, 2024
4fa86e6
Add nccl bcast method and its unit tests.
manauref Feb 22, 2024
091cedb
Fixes to support parallelization in adiabatic elc model. Change some …
manauref Feb 22, 2024
245efed
Use the average of the sheath potential and density on lower and uppe…
manauref Feb 22, 2024
925d503
Minor changes for safer initialization of comms or lack thereof.
manauref Feb 22, 2024
603505c
Merge pull request #283 from ammarhakim/gk-g0-app-all-gather
JunoRavin Feb 22, 2024
8388d02
Add array_bcast to null_comm. Make null_comm_new call null_comm_inew.…
manauref Feb 23, 2024
49ceef1
Add comm stuff to gk_lapd reg test.
manauref Feb 23, 2024
4671dac
Change integrated moments to not scale by 1/J. J is needed because we…
akashukla Feb 23, 2024
7f07048
Make logic for int moments reduction match main app branch
akashukla Feb 23, 2024
fdcdedd
Merge branch 'gk-g0-app' into gk-g0-app-source-moms
akashukla Feb 23, 2024
fec26ac
After merging in the main branch all gather logic, Change gkyl_comm_a…
akashukla Feb 25, 2024
5fb1351
Regression tests run on multiple gpus. Adding a unit test which works…
akashukla Feb 26, 2024
c59e319
Unit test for integrated moms runs on gpus
akashukla Feb 26, 2024
be47b24
Merge branch 'gk-g0-app' into gk-g0-app-neut
akashukla Feb 26, 2024
b97dd11
Fix react regression tests to have bcs and source object syntax consi…
akashukla Feb 26, 2024
3f7dc4f
Merge pull request #285 from ammarhakim/gk-g0-app-source-moms
manauref Feb 26, 2024
d49e69e
Merge branch 'gk-g0-app' into gk-g0-app-neut
akashukla Feb 26, 2024
7eac330
Use omega_cfl instead of OmegaCfl. remove commented write calls
akashukla Feb 26, 2024
4e43dde
Remove some commented print statements
akashukla Feb 26, 2024
20d0f44
Removing num_mom from neut moment calculations to be consistent with …
tnbernard Feb 26, 2024
1e78c13
Merge branch 'gk-g0-app-neut' of https://github.com/ammarhakim/gkylze…
tnbernard Feb 26, 2024
79353c3
Removing some extraneous reg tests.
tnbernard Feb 26, 2024
ea74b0c
Correcting comment.
tnbernard Feb 26, 2024
3b57592
Fixing MPI broadcast test when running on only 1 rank. If the size of…
JunoRavin Feb 27, 2024
2069298
Trialing GK regression test refactor on 1x2v radiation test.
JonathanGorard Feb 27, 2024
a5fb3f2
Bug fix: the order of arguments to the boundary fluxes for the radiat…
JunoRavin Feb 27, 2024
5bc0be8
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
JunoRavin Feb 27, 2024
6eaa4e0
Refactored 1x2v collisional relaxation tests.
JonathanGorard Feb 27, 2024
4ed8e8c
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
JonathanGorard Feb 27, 2024
e62666d
Bug fix: only write out the field if we're updating the field. Turns …
JunoRavin Feb 27, 2024
1525bf7
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
JunoRavin Feb 27, 2024
4b77a7f
Small extension to radiation test to add gyrokinetic integrated momen…
JunoRavin Feb 27, 2024
81e2a2d
Realized this simulation should not compute the field energy (since t…
JunoRavin Feb 27, 2024
461de91
Juicing my commit stats by still forgetting to delete all the field e…
JunoRavin Feb 27, 2024
45a8307
Minor changes prior to merge. I could tell that the neut_react logic …
JunoRavin Feb 28, 2024
05fecf7
Wait nevermind the prim_moms in the projection object should only be …
JunoRavin Feb 28, 2024
a76eec9
Merge pull request #286 from ammarhakim/gk-g0-app-neut
JunoRavin Feb 28, 2024
d92a651
Fix step core boundary unit test. It was missing some of the new inpu…
akashukla Feb 28, 2024
0485590
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
akashukla Feb 28, 2024
718fc97
Fix step outboard unit test to have newere tok geo inputs
akashukla Feb 28, 2024
8e4044d
A few consistency enforcements across GK regression tests.
JonathanGorard Feb 28, 2024
b3568a0
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
JonathanGorard Feb 28, 2024
3fc7fd8
Two small corrections: the formatting in react seemed to be slightly …
JunoRavin Feb 28, 2024
7e02400
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
JunoRavin Feb 28, 2024
22cb10d
More cleanup. Changing recombination to do a loop over moments for di…
JunoRavin Feb 28, 2024
0d6f788
Change geometry initialization to match app more closely in radiation…
akashukla Feb 28, 2024
c4b8774
Responding to a comment that some numbers were being hard-coded in a …
JunoRavin Feb 28, 2024
510a7ef
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
JunoRavin Feb 28, 2024
2fd19fe
Merge branch 'gk-g0-app' into gk-g0-app-correct
JunoRavin Feb 28, 2024
8510273
Update radiation fit parameters file with Ar fits up to +12. Better f…
jRoeltgen Feb 28, 2024
c62c449
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
jRoeltgen Feb 28, 2024
ce73a9c
Adding MPI/NCCL to bgk 3x2v case that is just the LAPD case but with …
JunoRavin Feb 28, 2024
2c7d722
Merge branch 'gk-g0-app' into gk-g0-app-correct
JunoRavin Feb 28, 2024
4ecd9b9
Maximum atomic number of radiation_fit_parameters.txt had been writte…
jRoeltgen Feb 28, 2024
12e317c
Some utterly frivolous changes, but without them my GK regression scr…
JonathanGorard Feb 29, 2024
643b19d
Add a sample perlmutter gpu jobscript
Feb 29, 2024
b325a27
Refactored all GK conducting sheath boundary condition tests, plus ot…
JonathanGorard Mar 1, 2024
2134378
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
JonathanGorard Mar 1, 2024
9f90af4
Remove -n 8 from perlmutter jobscript
akashukla Mar 1, 2024
828ddfc
Remove --gpus 8 from perlmutter jobscript. it is not necessary
akashukla Mar 1, 2024
496c85c
Rename GKYL_GK_FIELD_ADIABATIC -> GKYL_GK_FIELD_BOLTZMANN, since in 2…
manauref Mar 2, 2024
4540fce
Fix missing factor of jacobgeo in the polarization weight for cdim=1.…
manauref Mar 2, 2024
4f16aec
Add missing MPI in gk_ion_sound_adiabatic_elc test. But I think I bro…
manauref Mar 2, 2024
cfe2792
Use the global jacobgeo in the polarization weight. The gk_sheath_1x2…
manauref Mar 2, 2024
3637278
Subtrackt the equilibrium electron density from the charge density. N…
manauref Mar 2, 2024
5dd418c
Modified .max_iter and .eps_err in the input struct for correct_maxwe…
dingyunl Mar 4, 2024
fa3ffe3
Use the input moments to project the maxwellian if it doesn’t converge.
dingyunl Mar 4, 2024
e15ba31
Decreased the .cfl_frac for numerical stability and extended the simu…
dingyunl Mar 4, 2024
0d9751a
Include NCCL flags to makefile.sample to get output into the shared m…
Maxwell-Rosen Mar 4, 2024
7144c8a
Fixed some mistakes
Maxwell-Rosen Mar 4, 2024
4166386
Merge pull request #294 from ammarhakim/gk-g0-app-nccl-fix
manauref Mar 4, 2024
f26824c
Merge pull request #291 from ammarhakim/gk-g0-app_boltzVadiabatic
JunoRavin Mar 5, 2024
8126c23
Trialing new clang-format linter on refactored GK regression tests.
JonathanGorard Mar 5, 2024
708ec02
Merge branch 'gk-g0-app' into gk-g0-app-correct
JunoRavin Mar 5, 2024
f4f6d40
Removing spurious test based on Akash's STEP calculations. We can do …
JunoRavin Mar 5, 2024
330d7a9
Typo in header of correct Maxwellian gyrokinetic
JunoRavin Mar 5, 2024
77bf6f3
Merge pull request #293 from ammarhakim/gk-g0-app-correct
JunoRavin Mar 5, 2024
78cadb1
Refactored LAPD 3x2v Cartesian test, plus enforced Jimmy's brace conv…
JonathanGorard Mar 5, 2024
0ef5bd8
Rename n for dens in gk_proj (we should not have long lived single le…
manauref Mar 5, 2024
d7b032b
Merge branch 'main' into gk-g0-app. Fixes some memory issues in fluid…
JunoRavin Mar 5, 2024
e961720
Trying to track down why the BGK test is not running and caught a sep…
JunoRavin Mar 6, 2024
d875a6c
BGK test is just a re-skinned LAPD test but using BGK collisions inst…
JunoRavin Mar 6, 2024
9c7019e
After some discussion with Jonathan and some reading he sent me about…
JunoRavin Mar 6, 2024
9b22e4c
Bug fix: the reduce method for computing the stable time step for BGK…
JunoRavin Mar 6, 2024
a56da4c
Bug fix: BGK memory release was improperly structured. We can't free …
JunoRavin Mar 6, 2024
92c40a4
The extension of the source object to add diagnostic moment capabilit…
JunoRavin Mar 6, 2024
37ca20d
Missed two small formatting fixes for the else { statements
JunoRavin Mar 6, 2024
a0eda5a
One array in correct_maxwellian_gyrokinetic.c was not being freed
JunoRavin Mar 6, 2024
450b94c
Updated traverse machine files for multi-gpu sims. Build time is just…
Maxwell-Rosen Mar 7, 2024
8b16e06
Update machine files for della-gpu to cuda 12 with nccl support for m…
Mar 7, 2024
b5b24cb
Upgrade the mirror unit tests to have mutli-gpu support
Maxwell-Rosen Mar 7, 2024
b02ad7e
Load mpi modules in stellar intel machine files
akashukla Mar 7, 2024
9f35386
Remove personal paths from stellar intel machine files
akashukla Mar 7, 2024
5bea0d5
Update ltx boltzmann electron rt with nccl
Maxwell-Rosen Mar 7, 2024
791781c
Update ltx rt with nccl
Maxwell-Rosen Mar 7, 2024
2853580
Add include paths to stellar intel configure file
akashukla Mar 7, 2024
8f193a8
Merge branch 'gk-g-3-app' of https://github.com/ammarhakim/gkylzero i…
akashukla Mar 7, 2024
d4fdae2
Remove unnecessary include paths in stellar intel configure file
akashukla Mar 7, 2024
530ed2e
Merge main -> gk-g0-app
manauref Mar 7, 2024
23f1b34
Fix CPU bgk_collisions for how it's used in the gk-g0-app. The GPU ve…
manauref Mar 8, 2024
bffa05c
Fix field flag in bgk reg test.
manauref Mar 8, 2024
e5ae7b4
Merge branch 'main' into gk-g0-app
manauref Mar 8, 2024
8e978c1
Reorganize fem_parproj unit test so it runs on both CPU and GPU at th…
manauref Mar 8, 2024
9483034
Bug fix: cleanup of BGK operator last night had an error that the div…
JunoRavin Mar 9, 2024
5ffba93
Fix name in gk_sheath_3x2v reg test
manauref Mar 11, 2024
10475bc
Clean the step regression tests up. There were several nearly identic…
akashukla Mar 11, 2024
b6aa60b
Forgot to git add latest changes to step test
akashukla Mar 11, 2024
e822b1d
Adding command to create share/adas dir in case it doesn't exist.
Mar 11, 2024
82cbcde
Adding -p flag to mkdir command for nest dirs.
tnbernard Mar 12, 2024
57609e3
Add protections in gkmaxwellian projection for when vtsq < 0 and the …
akashukla Mar 12, 2024
3ab583c
Add protection in proj maxwellian lab mom and prim mom (not gk kind)
akashukla Mar 12, 2024
7c61098
Add the basic exponential amplitude protection to project maxwellian …
akashukla Mar 12, 2024
3a129f4
All step geometry unit tests take < 1 minute to run
akashukla Mar 12, 2024
4ecd360
Merge branch 'gk-g0-app' of https://github.com/ammarhakim/gkylzero in…
akashukla Mar 12, 2024
27dcf8e
Fix typo in proj maxwellian cu file. Unit test ctest_proj_maxwellian_…
akashukla Mar 12, 2024
2adbfa4
Merge pull request #301 from ammarhakim/gk-g0-app_maxproj_protections
manauref Mar 12, 2024
adec1f1
Add a big array reduce unit test which fails on 1 GPU
Maxwell-Rosen Mar 12, 2024
d49cffc
Fixed typo by including an underscore
Maxwell-Rosen Mar 12, 2024
2c6b79b
Insert a multiplication by bmag, the gyrocenter coord jacobian, into …
manauref Mar 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
26 changes: 18 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,15 @@ ifeq (${USE_LUA}, 1)
CFLAGS += -DGKYL_HAVE_LUA
endif

# Read ADAS paths and flags if needed
USING_ADAS =
ADAS_INC_DIR = zero # dummy
ADAS_LIB_DIR = .
ifeq (${USE_ADAS}, 1)
USING_ADAS = yes
CFLAGS += -DGKYL_HAVE_ADAS
endif

# Build directory
ifdef USING_NVCC
BUILD_DIR ?= cuda-build
Expand All @@ -118,8 +127,8 @@ ifeq ($(UNAME), Darwin)
SHFLAGS_INSTALL = ${SHFLAGS} -install_name ${PREFIX}/gkylzero/lib/libgkylzero.so
endif

# Header files
HEADERS := $(wildcard minus/*.h) $(wildcard zero/*.h) $(wildcard apps/*.h) $(wildcard kernels/*/*.h)
# Header files
HEADERS := $(wildcard minus/*.h) $(wildcard zero/*.h) $(wildcard apps/*.h) $(wildcard kernels/*/*.h) $(wildcard data/adas/*.h)
# Headers to install
INSTALL_HEADERS := $(shell ls apps/gkyl_*.h zero/gkyl_*.h | grep -v "priv" | sort)
INSTALL_HEADERS += $(shell ls minus/*.h)
Expand All @@ -128,9 +137,9 @@ INSTALL_HEADERS += $(shell ls minus/*.h)
INCLUDES = -Iminus -Iminus/STC/include -Izero -Iapps -Iregression -I${BUILD_DIR} ${KERN_INCLUDES} -I${LAPACK_INC} -I${SUPERLU_INC} -I${MPI_INC_DIR} -I${NCCL_INC_DIR} -I${LUA_INC_DIR}

# Directories containing source code
SRC_DIRS := minus zero apps kernels
SRC_DIRS := minus zero apps kernels data/adas

# List of regression and unit tests
# List of regression and unit test
REGS := $(patsubst %.c,${BUILD_DIR}/%,$(wildcard regression/rt_*.c))
UNITS := $(patsubst %.c,${BUILD_DIR}/%,$(wildcard unit/ctest_*.c))
MPI_UNITS := $(patsubst %.c,${BUILD_DIR}/%,$(wildcard unit/mctest_*.c))
Expand Down Expand Up @@ -197,6 +206,10 @@ $(BUILD_DIR)/kernels/advection/%.c.o : kernels/advection/%.c
$(MKDIR_P) $(dir $@)
$(CC) $(CFLAGS) $(NVCC_FLAGS) $(INCLUDES) -c $< -o $@

$(BUILD_DIR)/kernels/bgk/%.c.o : kernels/bgk/%.c
$(MKDIR_P) $(dir $@)
$(CC) $(CFLAGS) $(NVCC_FLAGS) $(INCLUDES) -c $< -o $@

$(BUILD_DIR)/kernels/bin_op/%.c.o : kernels/bin_op/%.c
$(MKDIR_P) $(dir $@)
$(CC) $(CFLAGS) $(NVCC_FLAGS) $(INCLUDES) -c $< -o $@
Expand Down Expand Up @@ -233,10 +246,6 @@ $(BUILD_DIR)/kernels/maxwell/%.c.o : kernels/maxwell/%.c
$(MKDIR_P) $(dir $@)
$(CC) $(CFLAGS) $(NVCC_FLAGS) $(INCLUDES) -c $< -o $@

$(BUILD_DIR)/kernels/neutral_react/%.c.o : kernels/neutral_react/%.c
$(MKDIR_P) $(dir $@)
$(CC) $(CFLAGS) $(NVCC_FLAGS) $(INCLUDES) -c $< -o $@

$(BUILD_DIR)/kernels/pkpm/%.c.o : kernels/pkpm/%.c
$(MKDIR_P) $(dir $@)
$(CC) $(CFLAGS) $(NVCC_FLAGS) $(INCLUDES) -c $< -o $@
Expand Down Expand Up @@ -340,6 +349,7 @@ install: all $(ZERO_SH_INSTALL_LIB) ## Install library and headers
${MKDIR_P} ${INSTALL_PREFIX}/gkylzero/lib
${MKDIR_P} ${INSTALL_PREFIX}/gkylzero/bin
${MKDIR_P} ${INSTALL_PREFIX}/gkylzero/share
${MKDIR_P} ${INSTALL_PREFIX}/gkylzero/share/adas
# Headers
cp ${INSTALL_HEADERS} ${INSTALL_PREFIX}/gkylzero/include
./minus/gengkylzeroh.sh > ${INSTALL_PREFIX}/gkylzero/include/gkylzero.h
Expand Down
22 changes: 19 additions & 3 deletions Makefile.sample
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,22 @@ ifeq (${USE_MPI}, 1)
CFLAGS += -DGKYL_HAVE_MPI
endif

# Read NCCL paths and flags if needed (needs MPI and NVCC)
USING_NCCL =
NCCL_INC_DIR = zero # dummy
NCCL_LIB_DIR = .
ifeq (${USE_NCCL}, 1)
ifdef USING_MPI
ifdef USING_NVCC
USING_NCCL = yes
NCCL_INC_DIR = ${CONF_NCCL_INC_DIR}
NCCL_LIB_DIR = ${CONF_NCCL_LIB_DIR}
NCCL_LIBS = -lnccl
CFLAGS += -DGKYL_HAVE_NCCL
endif
endif
endif

# Read LUA paths and flags if needed
USING_LUA =
LUA_INC_DIR = zero # dummy
Expand All @@ -90,9 +106,9 @@ ifeq (${USE_LUA}, 1)
CFLAGS += -DGKYL_HAVE_LUA
endif

INCLUDES = -I${G0_INC_DIR} -I${LAPACK_INC} -I${SUPERLU_INC} -I${MPI_INC_DIR} -I${LUA_INC_DIR}
LIB_DIRS = -L${LAPACK_LIB_DIR} -L${SUPERLU_LIB_DIR} -L${MPI_LIB_DIR} -L${LUA_LIB_DIR}
EXT_LIBS = ${LAPACK_LIB} ${SUPERLU_LIB} ${MPI_LIBS} ${LUA_LIBS} -lm -lpthread -ldl
INCLUDES = -I${G0_INC_DIR} -I${LAPACK_INC} -I${SUPERLU_INC} -I${MPI_INC_DIR} -I${LUA_INC_DIR} -I${NCCL_INC_DIR}
LIB_DIRS = -L${LAPACK_LIB_DIR} -L${SUPERLU_LIB_DIR} -L${MPI_LIB_DIR} -L${LUA_LIB_DIR} -L${NCCL_LIB_DIR}
EXT_LIBS = ${LAPACK_LIB} ${SUPERLU_LIB} ${MPI_LIBS} ${LUA_LIBS} ${NCCL_LIBS} -lm -lpthread -ldl

all: rt_twostream

Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,10 @@ installed. Likewise, SuperLU **requires you to have cmake** installed.
in mind that on Apple Macs you do not need to do anything special to use
LAPACK/BLAS as it comes with the developer tools.

At this step, you can download ADAS data for neutral interactions by adding
the flag ```--use-adas=yes``` after the other dependencies to be
installed.

### Configuring

The (optional) configure step is used when we need to specify the use of
Expand Down
Loading
Loading