PluginsDevelopment
The NCrystal source repository contains two example plugins, which can serve as skeletons for how one is expected to create a given plugin. Notice that the development of plugins is a somewhat advanced feature, and it is assumed that all plugin developers have at least a basic familiarity with NCrystal, Python, and optionally C++ (C++ might not be needed if your plugin is merely serving up new NCMAT data files).
Of course, it is highly recommended to get in touch with the NCrystal developers before getting too far on developing your plugin, so we are aware of the plugin you are developing and can provide feedback, etc.
No matter what kind of plugin you want to provide, you must first pick a name for your plugin! For consistency, try to pick a name which is unique, clear, and concise, and consists only of characters in the english alphabet (a-z, both upper and lower cased characters allowed). Prefer CamelCasing. Example names of hypothetical (not actual) plugins could be CohInelasESS (coherent-inelastic models developed at ESS), SANSJC (small angle neutron scattering models, developed by James Chadwick), LoadHKLFmtXYZ (.hkl file format support, developed at XYZ institute), ESSLowTData (a plugin providing NCMAT files with special low-temperature materials for ESS). The name of your plugin can of course be changed later, but it is easier to simply pick a proper name to begin with.
Another important thing to do at this initial stage is to verify whether or not the license described in the
LICENSE file used in the examples below, namely the Apache 2.0 license, is indeed the license under which your plugin code is intended to be distributed. If possible, we recommend that you stick with this license, because then it will be straight-forward to integrate all or some of your contribution into the core NCrystal repository at any point in the future if deemed useful. A good reason for wanting to change the license could for instance be that the plugin is planned to be developed using code copied from existing projects under different license terms. If that is the case, make sure you update the LICENSE file in your repository accordingly (usually you can of course still keep any newly written code under the Apache 2.0 license, just make sure it is clear which files are under a separate license).
Of course, you should also make sure that the README.md file in your repository is updated appropriately, ensuring at a minimum that it describes what is provided by the plugin, with references for any physics equations, data sources, etc.
In the case where a plugin is merely intended to provide some extra static NCMAT files, writing and using the plugin is particularly easy - one must simply use the layout from the example DummyDataPlugin, and then dump the relevant NCMAT files into the data directory (instead of the example dummy.ncmat file which is already there). Of course, one should decide on a suitable plugin name and replace DummyDataPlugin with the chosen name, both in the names of directories and inside the pyproject.toml.
As a real example, take the actual plugin named WaterData (defined here in sub-directory of the ncrystal-extra repository). It contains several NCMAT files for light and heavy water at different temperatures, that were deemed too large to reside in the standard data library of NCrystal. Let us see what happens if we install the plugin via pip. For this particular plugin, it is published on PyPI, and we can simply install it with:
pip install ncrystal-plugins-WaterData
If the plugin had not been published on PyPI, we could have installed it directly from the online Git repository:
pip install "git+https://github.com/mctools/ncrystal-extra#subdirectory=pypkgs/ncrystal-plugin-WaterData"
Note that the #subdirectory=... part is needed only because the plugin in this case is not located at the root of the repository.
No matter which of the two commands above was used, we should now have the ncrystal-plugins-WaterData package installed (verify with pip list), and ncrystal will automatically notice and activate it.
Assuming we have also installed NCrystal itself, we can verify that the plugin is picked up correctly by running nctool --plugins which should give several lines of output, including a line for the WaterData plugin:
==> WaterData (dynamic from /some/where/../site-packages/ncrystal_plugin_WaterData/data)
If we also run nctool --browse we can see the data files provided by the plugin:
==> 32 files from "plugins" (/some/where/../site-packages/ncrystal_plugin_WaterData/data, priority=OnlyOnExplicitRequest):
plugins::WaterData/LiquidHeavyWaterD2O_T283.6K.ncmat
plugins::WaterData/LiquidHeavyWaterD2O_T293.6K.ncmat
<snip>
plugins::WaterData/LiquidWaterH2O_T600.0K.ncmat
plugins::WaterData/LiquidWaterH2O_T623.6K.ncmat
Which means that the new data files are now available for usage in NCrystal cfg-strings, but one must not forget the plugins::WaterData/ part of the filenames.
A plugin can also provide new fundamental capabilities to NCrystal, including for instance the addition of new scattering physics, or the support for new input data formats. In these cases, however, the plugin must be written in C++, and compiled into a binary (shared library) in order to work. This is obviously a bit more advanced, and before going down that route one might wish to carefully peruse the customphysics.cc example carefully, which is a small C++ application in which the C++ API is used directly to modify the physics. If you don' know C++, you should probably expect to first get at least a basic grasp of that language, or have colleague nearby who is ready to help out!
To get started, one should then go and look at
the example DummyPlugin which is intended to serve as a skeleton and example of how one can modify the scattering physics of a material, and how one most likely will have to add a new custom data section to NCMAT files in order to handle this. So the recommended way to start a new plugin is to first to pick a unique CamelCased name of your plugin (here we will assume you chose MyPluginName), then clone the ncrystal repository, and copy the contents below the examples/plugin folder to the root of your own new git repository (don't forget to give your repository a suitable name like perhaps ncplugin-MyPluginName or ncrystal-plugin-MyPluginName). Then edit the ncplugin_name.txt file and replace DummyPlugin with MyPluginName. You also need to edit the pyproject.toml to replace DummyPlugin with MyPluginName.
At this point you can already try to build and install the plugin. So assuming you are in a Python virtual environment with ncrystal installed, and your current working directory is the folder where you have the ncplugin_name.txt and other files, then you can (re)build and (re)install your plugin by simply doing:
pip install .
Verify that the plugin is there and providing a single data file with nctool --plugins and nctool --browse. Then you can verify that the plugin works by running:
ncrystal-pluginmanager --test MyPluginName
What this actually does is:
- It runs the test function defined in ./src/NCTestPlugin.cc, which is supposed to exercise the plugin functionality a bit, verify that all works as expected, or throw an exception if it does not.
- It verifies that all the NCMAT files provided by the plugin can be loaded (e.g. used in
createInfo,createScatter, etc.).
We can also try to inspect the single NCMAT file provided by this plugin, namely plugins::MyPluginName/somefile.ncmat:
nctool "plugins::MyPluginName/somefile.ncmat"
Giving a plot like (yeah, it is pretty nonsensical physics-wise at this point):
Now it is finally time to start working on the capabilities of the plugin!
For case 2, the actual features of the plugin itself is NCMAT data files provided in the data subdirectory of the plugin directory, and C++ code in the src subdirectory.
There are several C++ files in the src directory, but to develop a plugin for neutron scattering physics, only a few of those should be normally edited. It is actually recommended to leave the other files and most of the boiler plate code in the edited files intact, since that will make it easier to update the plugin in the case some of the required boiler plate in the future will have to be updated.
To make it easy to get started, the example DummyPlugin contains a silly example in which a plugin replaces the incoherent-elastic scattering physics of NCrystal with a dummy model. The model, implemented in the files NCPhysicsModel.hh and NCPhysicsModel.cc, can be activated by a custom section in NCMAT data. A factory implementation in NCPluginFactory.cc takes care of wrapping this physics model with the appropriate NCrystal interface class (a specialisation of the NCrystal::ScatterIsotropic class), and of combining the new physics model with existing physics models in NCrystal which are not replaced by the plugin. The factory is registered with NCrystal in the void NCP::registerPlugin() function.
In principle, that is all it takes: Modify NCPluginFactory.cc code to change the logic of how to combine your new physics model with existing models in NCrystal, and modify NCPhysicsModel.hh/.cc to implement your model instead of the dummy one. Of course, do not let the NCPhysicsModel.hh/.cc become huge and unmaintainable - if your model is complicated, you should probably break it down and add new helper classes and utilities in additional C++ files with helper functions and classes, and whatever is required by the plugin in question.
One additional modification is required though, and that is to modify the NCTestPlugin.cc code, which contains the function which is called when the plugin is tested via ncrystal-pluginmanager --test MyPluginName. It should of course be updated so that it exercises the features and data files related to the actual plugin being developed, and should in particular verify that the plugin is actually loaded, replaces exactly the scattering physics it is meant to replace (and nothing else), and that it has expected effects on scattering cross sections and scattering outcomes.