So many updates

We are pleased to release the 1.6.0 version of ncempy after a large number of updates to the package. Please see below for the major changes since the previous release:

• New features for the algo subpackage such as cross-correlation alignment, Fourier shift and shear operations, image moment calculations, image binning (simple change size)
• Major updates to mutlicorr for cross-correlation alignment with upsampling.
• New feature in eval.stack_align to use multicorr alignment against a stack of images.
• Capability of calculating a line_profile at arbitrary angles and with different widths. Look at eval.line_profile function
• Lots of updates to the io plugins. All plugins now have the simplified xxxReader() command for very simple data loading. Most plugins provide getMemmap() functionality to assist in analyzing data too large to fit in memory.
• Addition of example notebooks for some image review and analysis tasks. More will be added in future releases
• Testing infrastructure completely overhauled and changed to pytest. The tests actually work now!
• Fix a major bug with dm.py and the on_memory=True option where the data was lost if the file was closed.
• New visualization routines in the viz submodule. Easy image FFT display, image contrast based on standard deviation, and a simple 3D image viewer to assist in looking at time series or 3D volumes
• Test data is now available in the data directory. Tests will run locally without any additional downloads. Also provides example data for users to test things out.
• Remove h5py_cache dependency since h5py now provides this functionality using rdcc_ keywords. Thanks to @tschoonj for the pull request!

I probably forgot some things since its been so long since the last update. Those are the major updates though.

As always, if you find a problem please submit an issue so it can get solved quickly.

New edstomo module and io features added

We are pleased to release the 1.5.0 version of ncempy. Please see the major changes below

This version has one major addition from @ZGainsforth providing methods for reconstructing STEM-EDS tomography maps. This adds some extra dependencies and example data files. To install this feature you need use pip with the following command:
pip install “.[edstomo]”

io/dm.py and io/mrc.py now include a getMemmap() function for accessing large datasets on disk without loading the entire data set into memory. 4D-STEM data and large movie files from the CETA2 are two examples where this can be a very useful function. More memmap functions will be included in future releases as data sets get larger and larger.

emd.py can now write datasets with keyword arguments passed directly to h5py. This allows compression and other features to be used transparently. Also emd.py can now retrieve meta data from an emd group without loading the entire data set. Check out the get_emddims function. Thanks again, @ZGainsforth!

More attention to the algorithms and evaluation routines is planned for the next release as the io module seems to be stabilizing without the need for major fixes or features.

Any feedback is welcome. Please add issues to the github page if you find errors or want new features added.

Bug fixes for large MRC and DM files

This release incorporates two fixes to the mrc.py and dm.py file modules. There were two instances where very large files could not be read due to overflows in numpy integer32 registers.

Minor bug fixes

This is a bug fix release. It mainly fixes an issue with the getSlice() function of ncempy.io.dm.py. This function previously always returned the first slice of a 3D array if the on_memory option was True. This is now fixed.

Velox support, new licensing and more!

Ncempy is now at version 1.4.0. Here are the major changes:

• Added support for Velox EMD files. We currently only support images (2D) and image series (3D). Check out ncempy.io.emdVelox
• Change to MIT licensing for the io package to permit ncempy.io to be used in more packages. We are trying to support the entire community to improve the ease of use of your own data.

Minor updates:

• Fixed a bug in ncempy.io.ser.serReader where only the first image was loaded and copied many times for time series data
• Updated documentation to use numpy docstrings. The entire package is not there yet, but that will be finished soon.
• Many more changes can be seen in the commit logs.

Please send any feedback, requests and bugs through the Github issues tracker. We will fix them asap.

Update for very large MRC files

A small bug was found in the mrc.py code which only allowed MRC files with a limited number of z slices. This update increases that substantially and is important for reading very large MRC files.

MRC reader, improved commenting/examples, simple reader functions

This release adds a some significant changes to the ncempy.io capabilities.

• Improved commenting including examples on how to use the io functions.
• New MRC reader class.
  • This is a ubiquitous open source file format used in the tomography community and is now used by Thermo Fischer (FEI) microscopes for TEM movies written from TIA and Velox.
  • MRC reader can read the full header of the MRC file. Most important it returns the pixel size (always in Angstroms)
  • Includes mrcReader function to conveniently read the entire data set with one function call.
  • Includes mrcWriter functionality to allow creating MRC files which can be read by IMOD, tomviz, Digital Micrograph, and many other programs.
• SER, DM and MRC classes now include a getSlice() function to read only 1 image from a large file.
• SER, DM and MRC include handy reader functions to easily read the entire file and metadata returned in a common format of dictionary keys.
  • For example: ser.serReader() can read EELS/EDS spectra line scans and spectrum maps
• I am in the process of integrating ncempy with CAMERA's Xicam project. A basic xicam.NCEM plugin is nearly completed to utilize ncempy's functionality to quickly look at S/TEM data.

Gatan DigitalMicrograph (DM3/DM4) reader

This release adds a new class called dm.py which reads all tags and data in Gatan's DigitalMicrograph (DM) format.

• Most tags are read properly
• 1D, 2D, 3D and 4D data sets can be loaded as Numpy arrays
• Automatically logs pixel size, unit names and other important tags
• Capability to manually open a file, parse tags and read data sets by separate functions
• Includes a handy dmReader() function to simplify all steps of reading data from a file
• Includes a flag (on_memory) to minimize disk access for distributed file systems by parsing the file entirely in memory (thanks @gonzalorodrigo!)

Some fixes to ringdiff and other fixes can be seen in the commits.

Release 1.1.2

Fixes to issues 3 and 4:

• saving dimensions in EMD in the right order
• corresponding adjustments in RingDiff

Bug fix in io.emd

Former it was possible to break opening an EMD in readonly, which did not have all recommended groups.