Python API to the plasma-state

[dlg0]

More to come ...

[elwasif]

There MAY be an easier path. Long time ago, the guys at Tech-X were interested in coding up a C++ interface top the PS. Alex Pletzer (I think) actually did this, and Doug incorporated the code into the Python script that generates the plasma state. The main idea is to put a simple F77 wrapper around all PS calls, and then put a C and C++ interface around those F77 wrappers .

The code is in the old ORNL swim repo (which now lives in https://cswim.org/svn/cswim/ips/branches/combined-code) in state/src/ccps_lib and state/src/ccps_test. The code in ccps_lib is augmented greatly when the state is actually built (see it in its full glory on Edison in /project/projectdirs/m876/elwasif/trunk/components/state/src/ccps_lib )

Now I managed to build this coder and the associated test code on Edison (I had to fill in some gaps for some missing header files, so it may not be the 100% correct). When I run the test I get some weird output that suggests some MPI functionality is being invoked and I'm not really sure if the test is succeeding or not.

Soooooooo, if we convince ourselves that this C/C++ interface actually works against the current incarnation of the PS (it was done over 5 years ago from some of the dates in the source files), it should be relatively straightforward to build a Python wrapper around it (using swig http://www.swig.org/). For that we will need to code up some meaningful tests that exercise the routines we care about. We may have to deal with some issues related to the use of static libraries as the basis for a Python module, but a quick google search suggests this may not be totally insane.

[parkjm]

ccps_lib is a part of public release of PS. Not sure how often components/state is updated. It might be good idea to start with http://w3.pppl.gov/ntcc/PlasmaState/. @dlg0 already built most kind of stuffs including NTCC, ...

[elwasif]

@parkjm Indeed. I've just rebuilt the PS version from PPPL on edison (I needed to add -fpic flag for Python's sake), and the included ccps_lib builds fine. I plan on using this version to build the Python module.

[elwasif]

The (very alpha) PS python module, based on the ntcc plasma state code is in /project/projectdirs/atom/users/elwasif/plasma_state_build/pyplasmastate on edison. This one uses the Intel build of the NTCC code, but that shouldn't matter (I think).

I can import the module and create a new instance of the PS from Python. Data movement in/out of the underlying Fortran code "may" need more work, especially if we want this to work seamlessly with Numpy (the C++ PS interface uses its own multiarray class).

import pyplasmastate
g = pyplasmastate.PlasmaState("Test", 1)
dir(g)
['class', 'del', 'delattr', 'dict', 'doc', 'format', 'getattr', 'getattribute', 'hash', 'init', 'module', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'swig_destroy', 'swig_getmethods', 'swig_setmethods', 'weakref', 'alloc', 'deriveMhdEq', 'finishSpecies', 'getCharSize', 'getDims', 'getHandle', 'getState', 'getVersionId', 'hashDiff', 'interp1d', 'interp2d', 'readMachDescr', 'readShotConfig', 'setData', 'setFusionSpecies', 'setImpuritySpecies', 'setNeutralBeamSpecies', 'setRFMinoritySpecies', 'setThermalSpecies', 'store', 'this', 'updateEquilibrium', 'updateHashCodes', 'verifyMhdEq', 'writeGeqdsk', 'writeMachDescr', 'writeShotConfig', 'writeUpdateFile']

[parkjm]

@elwasif

Great!!!

Could you please make PS module access the plasma state variables (access to ps%ns, ...)?

The component helpers generally look like (for example https://github.com/ORNL-Fusion/ips-fastran/blob/master/src/zgenray.py )

Input helper:
ps = pyplasmastate.PlasmaState(ps_filename, 1)
zeff = ps["Zeff"] % Copy array ps%Zeff to zeff
Ti = ps["Ti"] % ...
calculate something using zeff, Ti ... needed for the code input files

Output helper:
Ti = ... %extract data from code output
ps["Ti"] = Ti % put Ti to plasma state
ps.store()

Then perfect, I think!!

[elwasif]

@parkjm Right now, almost everything there is pretty much auto-generated by SWIG. I'm still exploring what can be done using this starting point, and then see how best to modify it and/or build something on top to make it more intuitive.

[parkjm]

@elwasif,
hope to accelerate this task.... I have a visitor student from S. Koera, who is working on making ASTRA component, especially for KSTAR. If the python PS API is available, I believe he can make very rapid progress.

[elwasif]

@parkjm Well, I think I can get something in there (Alpha/Beta quality) for get/set functionality. The interface may not be exactly what you have in mind, but we will be able to change that incrementally. Having access to the other routines (interpolation, ..etc) is a different story, but I suspect you're not using them anyway.

[parkjm]

@elwasif Thanks. Yes, we can change the interface incrementally. I'm not a big fan of the PS interpolation API, but many of IPS components in the SWIM repo rely on the interpolation API. I think the interpolation API is one of the key elements. Otherwise, I'm actually using most of the routines in PS (through calling a general utility fortran code -- sure in ugly fashion).

[elwasif]

@bernhold @batchelordb The initial version of the Plasma state Python API is now on Edison in /project/projectdirs/atom/users/elwasif/plasma_state_build/pyplasmastate

Some notes

• This is a direct wrapping of the C++ interface defined in the ccps_lib thatis part of the standard PS distribution
• This provides a subset of the functionality available in the PS F90 module
• This uses it's own multi dimensional array class, and a vector class that wraps the C++ standard template library vector.
• Numpy arrays are not currently supported (so you cannot pass a Numpy array into the PS to set a variable, or get a variable back as a Numpy array)
• The file teststate.py has a direct translation of the C++ test code in ccps_test/ccps_test.cxx code and it runs (there's some dicrepancy in the standard output, and we will need more testing to make sure the functionality is corect).
• The interface is currently (naturally) very C++'ish, we'll probably want to build something more Pythonic on top of this one.
• One will need to add /project/projectdirs/atom/users/elwasif/plasma_state_build/pyplasmastate to LD_LIBRARY_PATH and PYTHONPATH for this to work
• The list of supported methods is (not all of them tested in the teststate.py file)
  'alloc', 'deriveMhdEq', 'finishSpecies','getCharSize', 'getDataDouble', 'getDataDoubleVec', 'getDataInt','getDataIntVec','getDataString','getDataStringVec','getDims', 'getHandle','getState','getVersionId','hashDiff','interp1d','interp2d','readMachDescr','readShotConfig',
  'setData', 'setFusionSpecies','setImpuritySpecies','setNeutralBeamSpecies','setRFMinoritySpecies',
  'setThermalSpecies','store', 'this', 'updateEquilibrium', 'updateHashCodes', 'verifyMhdEq',
  'writeGeqdsk', 'writeMachDescr', 'writeShotConfig', 'writeUpdateFile'

[parkjm]

Cool!! Could you please add getitem and setitem to alias getData() and setData()?

[elwasif]

@parkjm My next step is to try and add a dictionary-like interface on top of this, but I'd prefer if this layer was better tested before we slap another layer on top of it - especially using the multiarray and vector classes to make sure we have correct functionality.

[parkjm]

@elwasif I was able to replace a fortran code pstool by pstool.py using pyplasmastate. The really great thing is that now we can handle plasma state inside of the component wrapper!! The getData and setData are not too bad but having more intuitive data access like ps["Zeff"] will be super great. Thanks.

[elwasif]

@parkjm I'm working on it. I think I have it working for scalars and one-D vectors, and now i'm looking into higher dimension data and a couple of other calls to the PS

[dlg0]

@parkjm Is the Python PS API base pstool replacement at the point where I can use it? I'm trying to alter the ips-fastran/src/zgenray.py to read the abridged species list rather than the full one, but it seems I'd need to look at the pstool source to do it. When possible, perhaps you could make the pstool replacement available?

[parkjm]

@dlg0 Better to wait a little bit more until @elwasif release the first version of pyplasmastate.py. Then you can do it in minutes :) At that point, I'm going to throw away pstool.f90. (BTW, you can find pstool.f90 in /project/projectdirs/atom/users/parkjm/utils/pstool.)

[jcandy]

Gents. In the FACETS project they were critical of the "native" interface to plasma state data although the file format itself they liked just fine. I think this is an important issue that we may want to discuss as a group soon.

[batchelordb]

What was the criticism of the "native" interface? Was it just that it was in Fortran and not C++?

[jcandy]

:-)

Its been a while but I think it was an issue of portability and/or difficulty to build.

[batchelordb]

That is a legitimate criticism

[elwasif]

A new version of the PS Python interface is now in place. This version is built on top the earlier module and presents a more Pythonic interface (Dictionary-like access, support for native 1-D Python vector-like data strctures, and support for Numpy arrays). The code is on Edison in
/project/projectdirs/atom/users/elwasif/plasma_state_build/pyplasmastate/plasmastate.py and a port of the teststate.py to the new interface is in newtest.py .

Some comments

• Calls to set a Plasma state 1-D vector can take a list, tuple, or a Numpy 1-D array
• Calls to set a PS variable with higher dimension takes only Numy arrays
• Calls to get a PS variable ALWAYS return a Numpy array
• I couldn't find a test that exercises the 2D or higher dimension PS variables, so any help there would be appreciated.

[parkjm]

👍

Observations:

• for string, __setitem__ works, while __getitem__ does not
• multl-dimensional variable need to be transposed:

For example
ps["ts"][0] ==> 0th thermal species temperature
ps["ts"][1] ==> 1sth thermal species temperature
In the current implementation, ps["ts"].transpose()[0] gives 0th thermal species temperature

• In the current implementation, variable access as lower case. We may better choose case-insensitive or case-sensitive.

[elwasif]

for string, setitem works, while getitem does not

This was actually a bug in the PPPL PS C++ code. It's fixed now (which begs another question, how do we contribute this bug fix and/or the whole Python interface back to PPPL?)

multl-dimensional variable need to be transposed:

I think I've fixed this now

In the current implementation, variable access as lower case. We may better choose case-insensitive or case-sensitive.

So the lower case is hard-wired in the f77 code, we can make the Python layer case insensitive

[parkjm]

1. Still multi-dimensional variable needs to be transposed.
2. ps["rho"] = numpy.linspace(0.0,1.0,nrho) raises message rho is not a Plasma State scalar

@dlg0 you have access to TRANSP repo. How do you report any bug? I guess correction in TRANSP repo populates to the NTCC library?

[elwasif]

@parkjm

Still multi-dimensional variable needs to be transposed.

Can you give me a simple example to use for testing

ps["rho"] = numpy.linspace(0.0,1.0,nrho) raises message rho is not a Plasma State scalar

I think you're using an older version of the module, this was an informational/debug message to keep track of variable lookup in the internal dictionaries, and it has been suppressed in the latest version. It should still work (even if you get this message). I wonder if you're also not getting the latest fix for the transpose issue

[parkjm]

O.K, no more diagnostic output. Fort test of muil-dimensional variable, print ps["Ts"][0] in your newtest.py that should output an array with length of 101.

[parkjm]

@elwasif So the routine below

from plasmastate import  *
from numpy import *

ps = PlasmaState("test", 1)
ps["nrho"] = 5
ps.alloc()
ps["rho"] = linspace(0.0,1.0,ps["nrho"])
ps["nspec_th"] = 2
ps.setThermalSpecies(-1, -1, 0)
ps.setThermalSpecies(1, 1, 2)
ps.finishSpecies()

print 'Access'
print ps["Ts"]
print ps["Ts"][0]

print 'Set all'
ps["Ts"] = array([[2.,2.,2.,2.],[1.,1.,1.,1.]])
print ps["Ts"]

print 'Set one'
ps["Ts"][0] = array([3.,3.,3.,3.])
print ps["Ts"]

is supposed to generate screen output

Access
[[ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]
[ 0.  0.  0.  0.]
Set all
[[ 2.  2.  2.  2.]
 [ 1.  1.  1.  1.]]
Set one
[[ 3.  3.  3.  3.]
 [ 1.  1.  1.  1.]]

The output is, however,

Access
[[ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]]
[ 0.  0.]
Set all
[[ 2.  2.]
 [ 2.  2.]
 [ 2.  2.]
 [ 1.  1.]]
Set one
Traceback (most recent call last):
  File "check.py", line 23, in <module>
    ps["Ts"][0] = array([3.,3.,3.,3.])
ValueError: could not broadcast input array from shape (4) into shape (2)

[elwasif]

It should be there, where's that test file you're using?

[elwasif]

I see, could it be that it's not allocated or intialized?

[dlg0]

It's in /project/projectdirs/atom/users/greendl1/diem_tsc_jm_pstest and ncdump shows that the nbion var has contents, so it has to be allocated and initialized I'd think. It is string data if that makes a difference?

[dlg0]

Actually that's the run directory. The files are in the work/nb__nubeam_6 subdirectory, i.e,

/project/projectdirs/atom/users/greendl1/diem_tsc_jm_pstest/work/nb__nubeam_6

[elwasif]

The problem seems to be with all vectors of strings (scalar strings seem to work fine).

[elwasif]

A quick Fortran question, what does the following code return in data?

subroutine psget_nbion(iobj, n1, data, ierr)
   ....
    integer, intent(in) :: n1 ! list of dimensions (if any)
    CHARACTER*(*), intent(in)  :: data(n1) ! to read to/write from
   ....
   data(1:n1) = ss % nbion
end subroutine psget_nbion

[dlg0]

I'm surprised that even compiles since my understanding of intent(in) was that the variable cannot be changed. If it runs I'd hope that data should be whatever it was upon entry. What does happen?

Dr. David L Green
Fusion & Materials for Nuclear Systems
Oak Ridge National Laboratory
+1-865-241-2752

---

From: Wael Elwasif [email protected]
Date: June 2, 2015 at 11:56:49 AM EDT
To: ORNL-Fusion/ips-atom [email protected]
Cc: Green, David L. [email protected]
Subject: Re: [ips-atom] Python API to the plasma-state (#2)

A quick Fortran question, what does the following code return in data?

subroutine psget_nbion(iobj, n1, data, ierr)
....
integer, intent(in) :: n1 ! list of dimensions (if any)
CHARACTER_(_), intent(in) :: data(n1) ! to read to/write from
....
data(1:n1) = ss % nbion
end subroutine psget_nbion

Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-107998988.

[elwasif]

That's what I'm trying to figure out (this is the code that ends up causing the seg fault). I'm not sure if this is accessing an array of characters or an array of strings, and if the later then how big will each string be?. I tried to search in the PS code for the place where ss%nbion is allocated, but I can't find it.

[elwasif]

Actually the intent is out, I copied from the wrong routine

subroutine psget_nbion(iobj, n1, data, ierr)
    use plasma_state_mod, only : plasma_state_container, plasma_state, &
                                & load_ps_container, rspec
    implicit NONE
    integer, intent(in) :: iobj(*)  ! object handle
    integer, intent(in) :: n1 ! list of dimensions (if any)
    CHARACTER*(*), intent(out)  :: data(n1) ! to read to/write from
    integer, intent(out) :: ierr ! error code (0=OK)
    ! local
    type (plasma_state_container) :: ptr
    type (plasma_state), pointer  :: ss

    ierr = 0
    call ccps_argcheck('psget_nbion', iobj, ierr)
    if(ierr.ne.0) return
    ! proceed...
    call load_ps_container(iobj,ptr)
    ss => ptr % ps_ptr
    ! set
    data(1:n1) = ss % nbion
end subroutine psget_nbion

[dlg0]

So I've had a look in

/project/projectdirs/atom/atom-install-edison/transp-source/transp/codesys/source/plasma_state_kernel

and found at least where ss%nbion is allocated if you haven't already ...

greendl1@edison10:/project/projectdirs/atom/atom-install-edison/transp-source/transp/codesys/source/plasma_state_kernel> egrep -ri nbion * | grep -i allocate
plasma_state_copy_3.f90:  IF( ((ALLOCATED(ss2%nbion)).AND.(ALLOCATED(ss%nbion))).AND. &
plasma_state_copy_3.f90:  IF( ((ALLOCATED(ss2%nbion_trace)).AND.(ALLOCATED(ss%nbion_trace))).AND. &
plasma_state_empty_alloc_3.f90:  IF(ALLOCATED(ss%nbion)) then
plasma_state_empty_alloc_3.f90:             " ?plasma_state_empty_alloc: nbion already allocated with size:"
plasma_state_empty_alloc_3.f90:     ALLOCATE(ss%nbion(ss%nbeam))

[elwasif]

So is it an array of characters (i.e. a single string), or an array of strings?

[dlg0]

From

/project/projectdirs/atom/atom-install-edison/transp-source/transp/codesys/source/plasma_state_kernel/plasma_state_definition_mod.f90

 852      CHARACTER*32, DIMENSION(:), ALLOCATABLE :: nbion
 853        !name of species injected by each beam.
 854        !Possible values are: "H", "D", "T", "He3", and "He4" at present.
 855        !DMC Apr 2010: added: "Ne", "Ar", "Kr", "Xe"
 856        !Each beam can inject one species.  Multiple beams (or usually
 857        !all beams) can inject the same species.
 858        !the SNBI species list will be constructed from nbion elements;
 859        !in many experiments this will consist only of the single element "D".
 860        !N  nbion(nbeam)

So it is an array of allocatable length, with each entry being 32 characters.

[elwasif]

So when you capture this value using the statement from above

data(1:n1) = ss % nbion

Then does one know what the memory layout of data will be (since it'll need to be accessed from C++)?

[dlg0]

Any progress on this?

[elwasif]

On 06/25/2015 09:07 AM, David L Green wrote:

Any progress on this?

—
Reply to this email directly or view it on GitHub
#2 (comment).

I'm stumped with Fortran-ish string array issues that I'm not sure how
to fix. Maybe you can help?

Wael R. Elwasif, PhD.
Research Staff Member
Computer Science Research Group
Oak Ridge National Laboratory
P.O. Box 2008, Bldg. 5600, MS 6016
Oak Ridge, TN 37831-6016

Office : (865)241-0002
Fax: (865)576-5491

[parkjm]

Revisiting two issues @dlg0 raised:

• interp1d
• get string data

interp1d

1D profile, like Ti(dm1_nrho), works fine

tmp = ps.interp1d("ti",0,rho)

Array of 1D profile , like nbeami(dim_nspec_beam, dm1_nrho_nbi), does not work.

tmp = ps.interp1d(“nbeami",0,rho,0)

Note that for nbeami, there needs to be another "last" argument .

In order to fix it, interp1d routine in plasmastate.py

if index:
    ret_vec = self._ps.interp1d(nm, deriv, vec, index)
else
    ret_vec = self._ps.interp1d(nm, deriv, vec)

needs to be changed something like

if index!=None:
...

get string data

There is a bug in ccps_lib/cxxps_getset.cpp, line 472. There should be & in front of resFlat like:

res[i].assign(&resFlat[i * sz], sz);

@elwasif , @dlg0

NTCC library with the bug fix was built on /project/projectdirs/atom/users/parkjm/ntcc-intel. Test cases are available on /project/projectdirs/atom/users/parkjm/pyps with a quick fix for interp1d.

The python API is now stable, it would be nice for @elwasif to put them to somewhere in the git repo.

[elwasif]

@parkjm I'm not sure I get why

if index :

would give different result from

if index != None:

[elwasif]

@parkjm Similar code is also used in

    def interp2d(self, nm, deriv1, deriv2, x1s, x2s, index = None):
...
        if index:
            ret_vec = self._ps.interp2d(nm, deriv1, deriv2, x1s_vec, x2s_vec,
                                        index)
        else:
            ret_vec = self._ps.interp2d(nm, deriv1, deriv2, x1s_vec, x2s_vec)
'''

So that doesn't work?

[parkjm]

In order to access the first beam ion species,

ps.interp1d(“nbeami",0,rho,0)

, which should call

self._ps.interp1d(nm, deriv, vec, index)

rather than

self._ps.interp1d(nm, deriv, vec)

[parkjm]

Same for interp2d.

[elwasif]

@parkjm I see.

[elwasif]

@parkjm I'm unable to access the files in /project/projectdirs/atom/users/parkjm/pyps (I'm still getting the error in arrays of strings after applying your patch to my local build)

[parkjm]

@elwasif the file permission fixed...

[elwasif]

@parkjm Where do you have the sources and the binaries for the plasma state build that you use?

[parkjm]

@elwasif /global/project/projectdirs/atom/users/parkjm/ntcc I added only one character.

[elwasif]

This simple program still fails for me using your version

from plasmastate import *
ps = PlasmaState("ips",1)
ps.read("ips-state.nc")
print ps["NBION"]

ips-state.nc is taken from an IPS run (its at /project/projectdirs/atom/users/elwasif/plasma_state_build/pyplasmastate)

[parkjm]

@elwasif Looks O.K. now. Let me know if it still does not work.

[elwasif]

This version work (at least for the case that was giving me segfault before). I've applied the change to my local build, which uses PS 2.041. You're using 2.042. It mabe a good idea to update everything to 2.042, but that would require re-building the ntcc libs, Nubeam, SWIM wrappers, ..etc. Any thoughts @dlg0 ???

[parkjm]

IPS-FASTRAN has been updated with the python-API of plasma state (devel branch, ORNL-Fusion/ips-fastran@9595be2). So far no difference between fortran and python-API for the complicated use of the plasma state file in IPS-FASTRAN. I'm sure that this great new capability is ready to use (and must to use) for the new component development. Thanks again to @elwasif!

[transp]

The bug fix to cxxps_getset.cpp is now in TRANSP svn trunk. Will try to get it pushed to the NTCC repo version ASAP.

Thanks, Johan