Example files for each sub-reader

README.md

@@ -30,7 +30,7 @@ within the field of X-ray photoelectron spectroscopy into a standardized represe
 [NeXus](https://www.nexusformat.org/) application definition [NXmpes](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html#nxmpes).
 
 ## Docs
-Extensive documentation of this pynxtools plugin is available [here](fairmat-nfdi.github.io/pynxtools-xps/). You can find information about getting started, how-to guides, the supported file formats, how to get involved, and much more there.
+Extensive documentation of this pynxtools plugin is available [here](https://fairmat-nfdi.github.io/pynxtools-xps/). You can find information about getting started, how-to guides, the supported file formats, how to get involved, and much more there.
 
 ## Contact person in FAIRmat for this reader
 Lukas Pielsticker

docs/explanation/appdefs.md

@@ -1 +1 @@
-# The NeXus application definitions: NXmpes and NXxps
+# The NeXus application definitions: NXmpes <!-- and NXxps-->

docs/explanation/contextualization.md

@@ -4,6 +4,6 @@ Conceptually, mapping between representations of concepts and instance data is a
 
 In pynxtools-xps, the mapping from the vendor format is a two-step process:
 1) First, each information piece is parsed from the experiment- and vendor-specific and assigned a name that describes what the reader developer thinks it semantically means. This naming can come from documentation of the original data, existing key-value infrastructure in the data file, or from domain knowledge of the reader developer. All data and metadata items are internally stored as a flat list of dictionaries, with each dictionary containing all information about a single XP spectrum.
-2) This list of dicts is then mapped onto either the ([NXmpes](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html) NeXus application definition or its specialization [NXxps](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXxps.html)). For this, a JSON config file is used that provides a concept map from the originally assigned keys towards the groups, fields, and attributes in the NeXus standard. Such transformations are configured via the respective files in the [*config*](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/config) directory of pynxtools-xps.
+2) This list of dicts is then mapped onto either the ([NXmpes](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html) NeXus application definition <!-- or its specialization [NXxps](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXxps.html)) -->. For this, a JSON config file is used that provides a concept map from the originally assigned keys towards the groups, fields, and attributes in the NeXus standard. Such transformations are configured via the respective files in the [*config*](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/config) directory of pynxtools-xps.
 
 Upon parsing, the XPS reader uses the config file to map the (meta-)data to a *template* which follows the NeXus application definitions. It also takes metadata provided through additional means (i.e., an electronic lab notebook (ELN) file) to fill in missing required and recommended fields and attributes in the application definition that were not provided in the raw data fikes. It is this *template* variable from which core functions like *convert.py* of the pynxtools write the actual NeXus/HDF5 file. The latter tool is also referred to as the dataconverter of [pynxtools](https://github.com/FAIRmat-NFDI/pynxtools).

docs/how-tos/build_a_reader.md

@@ -54,4 +54,4 @@ You should add test data and add your reader to the `test_params` in the `test_r
 
 [NXmpes](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html)
 
-[NXxps](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXxps.html)
+<!-- [NXxps](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXxps.html) -->

docs/index.md

@@ -6,7 +6,7 @@ hide: toc
 
 pynxtools-xps is a free, and open-source data software for harmonizing X-ray photolectron spectroscopy data and metadata for research data management using [NeXus](https://www.nexusformat.org/), implemented with the goal to make scientific research data FAIR (findable, accessible, interoperable and reusable).
 
-pynxtools-xps, which is a plugin for [pynxtools](https://github.com/FAIRmat-NFDI/pynxtools), provides a tool for reading data from various propietary and open data formats from technology partners and the wider XPS community and standardizing it such that it is compliant with the NeXus application definitions [`NXmpes`](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html) and [`NXxps`](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXxps.html), which is an extension of `NXmpes`. pynxtools-xps is devloped both as a standalone reader and as a tool within [NOMAD](https://nomad-lab.eu/), which is the open-source data management platform for materials science we are developing with [FAIRmat](https://www.fairmat-nfdi.eu/fairmat/).
+pynxtools-xps, which is a plugin for [pynxtools](https://github.com/FAIRmat-NFDI/pynxtools), provides a tool for reading data from various propietary and open data formats from technology partners and the wider XPS community and standardizing it such that it is compliant with the NeXus application definition [`NXmpes`](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html) and <!--[`NXxps`](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXxps.html), which is an extension of `NXmpes` -->. pynxtools-xps is developed both as a standalone reader and as a tool within [NOMAD](https://nomad-lab.eu/), which is the open-source data management platform for materials science we are developing with [FAIRmat](https://www.fairmat-nfdi.eu/fairmat/).
 
 pynxtools-xps solves the challenge of using heterogeneous and unfindable data formats which is common in X-ray Photoelectron Spectroscopy. In addition, it provides an interface for writing readers for different file formats to be mapped to NeXus.
 
@@ -42,7 +42,7 @@ How-to guides provide step-by-step instructions for a wide range of tasks, with
 The explanation section provides background knowledge on the implementation design, how the data is structured, how data processing can be incorporated, how the integration works in NOMAD, and more.
 
 - [Design principles and implementation](explanation/implementation.md)
-- [NXmpes and NXxps](explanation/appdefs.md)
+- [NXmpes](explanation/appdefs.md) <!-- - [NXmpes and NXxps](explanation/appdefs.md) -->
 - [How to map pieces of information to NeXus](explanation/contextualization.md)
 <!-- - [Data processing](explanation/data_processing.md) -->
 <!-- - - [NOMAD integration](explanation/nomad_integration.md) -->
@@ -55,6 +55,7 @@ The explanation section provides background knowledge on the implementation desi
 Here you can learn which specific measurement setups and file formats from technology partners pynxtools-xps currently supports.
 
 The reader decides which data parser to use based on the file extension of the files provided. For the main XPS files, the following file extensions are supported:
+
 - .spe, .pro: [Phi MultiPak](https://www.phi.com/surface-analysis-equipment/genesis.html#software:multi-pak-data-reduction-software/) files, propietary format of PHI Electronics
 - .sle: [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) files, propietary format of SPECS GmbH (1 and v4)
 - .xml: SpecsLab 2files, XML format from SPECS GmbH (v1.6)

docs/reference/phi.md

@@ -6,15 +6,20 @@ The reader supports [Phi MultiPak](https://www.phi.com/surface-analysis-equipmen
 
 The reader for the Phi data can be found [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/phi).
 
-Example data for this file format is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/phi).
+## .spe data (single spectrum):
 
-The example conversion can be run with the following commands:
+Example data for this file format is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/phi).
 
-### For the .spe data (single spectrum):
+The example conversion can be run with the following command:
 ```console_
 user@box:~$ dataconverter SnO2_10nm.spe eln_data_phi.yaml --reader xps --nxdl NXmpes --output SnO2_10nm.spe.nxs
 ```
-### For the .pro data (profiling):
+
+### .pro data (profiling):
+Example data for this file format is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/phi).
+
+The example conversion can be run with the following command:
+
 ```console_
 user@box:~$ dataconverter SnO2_10nm_1.pro eln_data_phi.yaml --reader xps --nxdl NXmpes --output SnO2_10nm_1.pro.nxs
 ```

docs/reference/scienta.md

@@ -1,18 +1,33 @@
 # Data from Scienta Omicron instruments
 
-The reader supports reading data exported as .txt from Scienta Omicron [Scienta Omicron](https://www.scientaomicron.com/en/) instruments. <!--,  both as .txt as well as IGOR files.--> 
+The reader supports reading data exported as .txt from Scienta Omicron [Scienta Omicron](https://www.scientaomicron.com/en/) instruments. <!--,  both exported as .txt as well as ibw ([Igor Pro](https://www.wavemetrics.com/products/igorpro) binary wave) files.--> 
+
+The reader for the Scienta data can be found [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/scienta).
+
+## .txt data
 
 <!-- How is this data structured --> 
 
-The reader for the Scienta data can be found [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/scienta).
+Example data for the Scienta TXT reader is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/scienta/txt).
+
+The example conversion can be run with the following command.
+
+```console
+user@box:~$ dataconverter Cu-HHTP_*.txt eln_data.yaml --reader xps --nxdl NXmpes --output Cu-HHTP.txt.nxs
+```
+
+<!-- 
+## .ibw ([Igor Pro](https://www.wavemetrics.com/products/igorpro) binary wave) data
+
+How is this data structured
 
-Example data is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/scienta).
+Example data for the Scienta Igor reader is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/scienta/ibw).
 
 The example conversion can be run with the following command.
 
 ```console
-user@box:~$ dataconverter Cu-HHTP_*.txt eln_data.yaml --reader xps --nxdl NXmpes --output Cu-HHTP.nxs
+user@box:~$ dataconverter Cu-HHTP_*.ibw eln_data.yaml --reader xps --nxdl NXmpes --output Cu-HHTP.ibw.nxs
 ```
 
 ## Acknowledgments
-We thank Dr. Alexei Nefedov from [KIT](https://www.ifg.kit.edu/21_1296.php) for providing the example data set.
+We thank Dr. Alexei Nefedov from [KIT](https://www.ifg.kit.edu/21_1296.php) for providing the example data set.--> 

docs/reference/specs.md

@@ -6,15 +6,39 @@ The reader supports [SpecsLabProdigy](https://www.specs-group.com/nc/specs/produ
 - .xml: SpecsLab 2files, XML format from SPECS GmbH (software version: v4.63 tested, other versions also work)
 - .xy: SpecsLabProdigy export format in XY format (including all export settings)
 
-<!-- How is this data structured --> 
-
 The readers for the SPECS data can be found [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/specs).
 
-Example data for the SLE reader is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/specs).
+## .sle data
+
+<!-- How is this data structured --> 
+
+Example data for the SLE reader is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/specs/sle).
 
 The example conversion can be run with the following command.
 ```console
 user@box:~$ dataconverter --params-file params.yaml
 ```
 
-Note that the `params.yaml` file contains the `remove_align` keyword which is special for the SLE parser. It allows removal of alignment spectra that were taken during the experiment. For this example, it considerably speeds up the conversion.
+Note that the `params.yaml` file contains the `remove_align` keyword which is special for the SLE parser. It allows removal of alignment spectra that were taken during the experiment. For this example, it considerably speeds up the conversion
+
+## .xml data
+
+<!-- How is this data structured --> 
+
+Example data for the SPECS XML reader is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/specs/xml).
+
+The example conversion can be run with the following command.
+```console
+user@box:~$ dataconverter In-situ_PBTTT_XPS_SPECS.xml eln_data.yaml --reader xps --nxdl NXmpes --output In-situ_PBTTT.nxs
+```
+
+## .xy data
+
+<!-- How is this data structured --> 
+
+Example data for the SPECS XY reader is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/specs/xy).
+
+The example conversion can be run with the following command.
+```console
+user@box:~$ dataconverter MgFe2O4.xy eln_data.yaml --reader xps --nxdl NXmpes --output MgFe2O4.nxs
+``` 

docs/reference/vms.md

@@ -4,30 +4,41 @@
 
 The reader supports VAMAS (.vms) files, the ISO standard data transfer format ([ISO 14976](https://www.iso.org/standard/24269.html)) for X-ray photoelectron spectroscopy. The data can be stored both in REGULAR (i.e, with an equally spaced energy axis) as well as IRREGULAR mode. The data was measured with and exported from [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/).
 
-The reader for the VAMAS format can be found [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/scienta).
+The reader for the VAMAS format can be found [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/vms).
 
 Example data is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/vms).
 
-The example conversion for the REGULAR file can be run with the following command:
+The example conversion for the REGULAR VAMAS file can be run with the following command:
 
 ```sh
 ! dataconverter \
 regular.vms \
 eln_data_vms.yaml \
 --reader xps \
 --nxdl NXmpes \
---output vms_regular_example.nxs \
+--output regular.vms.nxs \
 ```
 
-The example conversion for the IRREGULAR file can be run with the following command:
+The example conversion for the IRREGULAR VAMAS file can be run with the following command:
 
 ```sh
 ! dataconverter \
 irregular.vms \
 eln_data_vms.yaml \
 --reader xps \
 --nxdl NXmpes \
---output vms_irregular_example.nxs \
+--output irregular.vms.nxs \
+```
+
+The example conversion for the .txt export file can be run with the following command:
+
+```sh
+! dataconverter \
+vms_txt_export.txt \
+eln_data_vms_txt_export.yaml \
+--reader xps \
+--nxdl NXmpes \
+--output vms_txt_export.nxs \
 ```
 
 <!-- ## Data analysis in CasaXPS -->

examples/phi/README.md

@@ -1,18 +1,28 @@
-# Example for data from Phi VersaProbe 4 instruments
+# Data from Phi VersaProbe 4 instruments
 
-This is an example for [Phi MultiPak](https://www.phi.com/surface-analysis-equipment/genesis.html#software:multi-pak-data-reduction-software/) .spe (single spectra) and .pro (sputter profile / external parameter scan / ....) files, which is the propietary format of PHI Electronics used for their VersaProbe 4 instrumens. The Phi MultiPak software version that was used to measure this data is SS 3.3.3.2.1. 
-We thank Sebastian Benz and Dr. Joachim Sann from [Justus-Liebig-Universität Gießen](https://www.uni-giessen.de/de) for providing these example data sets.
+The reader supports [Phi MultiPak](https://www.phi.com/surface-analysis-equipment/genesis.html#software:multi-pak-data-reduction-software/) .spe (single spectra) and .pro (sputter profile / external parameter scan / ....) files, which is the propietary format of PHI Electronics used for their VersaProbe 4 instrumens. The Phi MultiPak software version that was used to measure this data is SS 3.3.3.2.1. 
 
-The example conversion can be run with the following commands.
+<!-- How is this data structured -->
 
-### For the .spe data (single spectrum):
+The reader for the Phi data can be found [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/phi).
+
+## .spe data (single spectrum):
+
+Example data for this file format is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/phi).
+
+The example conversion can be run with the following command:
 ```console_
 user@box:~$ dataconverter SnO2_10nm.spe eln_data_phi.yaml --reader xps --nxdl NXmpes --output SnO2_10nm.spe.nxs
 ```
-### For the .pro data (profiling):
+
+### .pro data (profiling):
+Example data for this file format is available [here](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples/phi).
+
+The example conversion can be run with the following command:
+
 ```console_
 user@box:~$ dataconverter SnO2_10nm_1.pro eln_data_phi.yaml --reader xps --nxdl NXmpes --output SnO2_10nm_1.pro.nxs
 ```
 
-## Contact person in FAIRmat for this example
-Lukas Pielsticker
+## Acknowledgments
+We thank Sebastian Benz and Dr. Joachim Sann from [Justus-Liebig-Universität Gießen](https://www.uni-giessen.de/de) for providing these example data sets.

examples/regenerate_examples.sh

@@ -18,33 +18,44 @@ function update_scienta_examples {
   echo "Update scienta examples"
   cd scienta/
   echo "Update .txt example"
-  dataconverter Cu-HHTP_*.txt eln_data.yaml --reader $READER --nxdl $NXDL --output Cu-HHTP.nxs
+  dataconverter Cu-HHTP_*.txt eln_data.yaml --reader $READER --nxdl $NXDL --output Cu-HHTP.txt.nxs
   cd ..
   echo
 }
 
-function update_sle_examples {
+function update_specs_examples {
   echo "Update SPECS examples"
+  cd specs/
   cd sle/
   echo "Update .sle example"
   dataconverter --params-file params.yaml
-  cd ..
+  cd ../xml
+  echo "Update .xml example"
+  dataconverter In-situ_PBTTT_XPS_SPECS.xml eln_data.yaml --reader $READER --nxdl $NXDL --output In-situ_PBTTT.nxs
+  dataconverter --params-file params.yaml
+  cd ../xy
+  echo "Update .xy example"
+  dataconverter MgFe2O4.xy eln_data.yaml --reader $READER --nxdl $NXDL --output MgFe2O4.nxs
+  cd ../..
   echo
 }
 
 function update_vms_examples {
   echo "Update VAMAS examples"
   cd vms/
   echo "Update REGULAR file conversion example"
-  dataconverter regular.vms eln_data_vms.yaml --reader $READER --nxdl $NXDL --output vms_regular_example.nxs
+  dataconverter regular.vms eln_data_vms.yaml --reader $READER --nxdl $NXDL --output regular.vms.nxs
   echo
   echo "Update REGULAR file conversion example"
-  dataconverter irregular.vms eln_data_vms.yaml --reader $READER --nxdl $NXDL --output vms_irregular_example.nxs
+  dataconverter irregular.vms eln_data_vms.yaml --reader $READER --nxdl $NXDL --output irregular.vms.nxs
+  echo "Update REGULAR file conversion example"
+  dataconverter irregular.vms eln_data_vms_txt_export.yaml --reader $READER --nxdl $NXDL --output vms_txt_export.nxs
   cd ..
   echo
 }
 
 
 update_phi_examples
 update_scienta_examples
-update_sle_examples
+update_specs_examples
+update_vms_examples

examples/scienta/README.md

@@ -6,7 +6,7 @@ This is an example for parsing the .txt data export of Scienta Omicron
 The example conversion can be run with the following command.
 
 ```console
-user@box:~$ dataconverter Cu-HHTP_*.txt eln_data.yaml --reader xps --nxdl NXmpes --output Cu-HHTP.nxs
+user@box:~$ dataconverter Cu-HHTP_*.txt eln_data.yaml --reader xps --nxdl NXmpes --output Cu-HHTP.txt.nxs
 ```
 
 ## Contact person in FAIRmat for this example