Skip to content

Commit

Permalink
Release ZFP 0.5.1
Browse files Browse the repository at this point in the history
  • Loading branch information
salasoom committed Mar 30, 2017
1 parent d2b1fd5 commit bf7c8be
Show file tree
Hide file tree
Showing 82 changed files with 2,220 additions and 880 deletions.
40 changes: 22 additions & 18 deletions API
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,9 @@ array:
zfp_type type = zfp_type_double;
zfp_field* field = zfp_field_3d(&a[0][0][0], type, nx, ny, nz);

For single-precision data, use zfp_type_float. Note that the high-level
API does not support integer arrays (zfp_type_int32 and zfp_type_int64).
Such arrays must be compressed via the low-level interface.
For single-precision data, use zfp_type_float. As of version 0.5.1, the
the high-level API also supports integer arrays (zfp_type_int32 and
zfp_type_int64). See FAQs #8 and #9 regarding integer compression.

Functions similar to zfp_field_3d exist for declaring 1D and 2D arrays.
If the dimensionality of the array is unknown at this point, then a generic
Expand All @@ -118,8 +118,8 @@ for more details on the meaning of these parameters):

// set compression mode and parameters
zfp_stream_set_rate(zfp, rate, type, dims, 0);
zfp_stream_set_precision(zfp, precision, type);
zfp_stream_set_accuracy(zfp, tolerance, type);
zfp_stream_set_precision(zfp, precision);
zfp_stream_set_accuracy(zfp, tolerance);

Note that only one of these three functions should be called. The return
value from these functions gives the actual rate, precision, or tolerance,
Expand Down Expand Up @@ -170,13 +170,15 @@ the same sequence of function calls as above, or by recording these
fields and setting them directly. Metadata such as array dimensions and
compression parameters are by default not stored in the compressed stream.
It is up to the caller to store this information, either separately from
the compressed data, or via the zfp_write_header and zfp_read_header calls.
These calls allow the user to specify what information to store in the
header, including a 'magic' format identifier, the field type and
dimensions, and the compression parameters (see the ZFP_HEADER_* macros).
the compressed data, or via the zfp_write_header and zfp_read_header calls,
which must precede the corresponding zfp_compress and zfp_decompress
calls, respectively. These calls allow the user to specify what
information to store in the header, including a 'magic' format identifier,
the field type and dimensions, and the compression parameters (see the
ZFP_HEADER_* macros).

In addition to this initialization, the bit stream has to be rewound to
the beginning:
the beginning (before reading the header and decompressing the data):

// rewind compressed stream and decompress array
zfp_stream_rewind(zfp);
Expand All @@ -190,15 +192,15 @@ ZFP LOW-LEVEL COMPRESSION AND DECOMPRESSION CODEC
For applications that wish to compress or decompress portions of an array
on demand, a low-level interface is available. Since this API is useful
primarily for supporting random access, the user also needs to manipulate
the bit stream (see inc/bitstream.h), e.g. to position the bit pointer
the bit stream (see include/bitstream.h), e.g. to position the bit pointer
to where data is to be read or written. Please be advised that the bit
stream functions have been optimized for speed, and do not check for
buffer overruns or other types of programmer error.

Like the high-level API, the low-level API also makes use of the zfp_stream
parameter object (see section above) to specify compression parameters and
storage, but does not encapsulate array metadata in a zfp_field object.
Functions exists for encoding and decoding complete or partial blocks, with
Functions exist for encoding and decoding complete or partial blocks, with
or without strided access. In non-strided mode, the uncompressed block to
be encoded or decoded is assumed to be stored contiguously. For example,

Expand Down Expand Up @@ -232,6 +234,8 @@ For partial blocks, e.g. near the boundaries of arrays whose dimensions
are not multiples of four, there are corresponding functions that accept
parameters (nx, ny, nz) to specify the actual block dimensions, with
1 <= nx, ny, nz <= 4. Corresponding functions exist for decompression.
Such partial blocks typically do not compress as well as full blocks and
should be avoided if possible.

To position a bit stream for reading (decompression), use

Expand All @@ -258,7 +262,7 @@ decompress only the first 256 bits of each block:
zfp_stream_set_params(zfp, 256, 256, maxprec, minexp);

This feature may be combined with progressive decompression, as discussed
further in the FAQ.
further in FAQ #13.


COMPRESSED ARRAYS
Expand Down Expand Up @@ -297,9 +301,9 @@ representation. By default the block size is restricted to a multiple of
2D arrays: 4-bit granularity
3D arrays: 1-bit granularity

For finer granularity, the BITSTREAM_WORD_TYPE macro needs to be set to a
type narrower than 64 bits, e.g. if set to uint8 the rate granularity
becomes 8 / 4^d bits in d dimensions, or
For finer granularity, the BIT_STREAM_WORD_TYPE macro needs to be set to a
type narrower than 64 bits during compilation of libzfp, e.g. if set to
uint8 the rate granularity becomes 8 / 4^d bits in d dimensions, or

1D arrays: 2-bit granularity
2D arrays: 1/2-bit granularity
Expand Down Expand Up @@ -381,7 +385,7 @@ Array dimensions (nx, ny, nz) can be queried using these functions:
uint size_z(); // nz

The array dimensions can also be changed dynamically, e.g. if not known
at declaration time, using
at time of construction, using

void resize(uint nx, uint ny, uint nz, bool clear = true);

Expand All @@ -405,7 +409,7 @@ CACHING
As mentioned above, the array class maintains a software write-back cache
of at least one uncompressed block. When a block in this cache is evicted
(e.g. due to a conflict), it is compressed back to permanent storage only
if it has previously been modified.
if it was modified while stored in the cache.

The size cache to use is specified by the user, and is an important
parameter that needs careful consideration in order to balance the extra
Expand Down
37 changes: 28 additions & 9 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
cmake_minimum_required(VERSION 3.1)
if(WIN32)
cmake_minimum_required(VERSION 3.4)
else()
cmake_minimum_required(VERSION 3.1)
endif()

# Fail immediately if not using an out-of-source build
if(CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_CURRENT_BINARY_DIR)
Expand All @@ -10,7 +14,7 @@ endif()
#------------------------------------------------------------------------------#
# Parse version number from zfp.h
#------------------------------------------------------------------------------#
file(READ ${CMAKE_CURRENT_SOURCE_DIR}/inc/zfp.h _zfp_h_contents)
file(READ ${CMAKE_CURRENT_SOURCE_DIR}/include/zfp.h _zfp_h_contents)
string(REGEX REPLACE ".*#define[ \t]+ZFP_VERSION_MAJOR[ \t]+([0-9]+).*"
"\\1" ZFP_VERSION_MAJOR ${_zfp_h_contents})
string(REGEX REPLACE ".*#define[ \t]+ZFP_VERSION_MINOR[ \t]+([0-9]+).*"
Expand Down Expand Up @@ -42,6 +46,15 @@ endif()
# Top level options
#------------------------------------------------------------------------------#

# Windows specific options
if(WIN32)
# Use this to get a usable export library when building a DLL on Windows
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)

# Silence extraneous Visual Studio specific warnings
add_definitions(-D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS /wd4146 /wd4305)
endif()

# Suggest C99
# Falls back to C89 if 99 is not supported
set(CMAKE_C_STANDARD 99)
Expand All @@ -50,9 +63,9 @@ include(CMakeDependentOption)

# Typically you'd always be able to enable shared libraries but default
# configurations with the Cray toolchain will explicitly disable shared lib
# supprot and only allow static libs. Making this a cmake_dependent_option
# will ensure that shared library support will be disabled if the system doesnt
# support it.
# support and only allow static libs. Making this a cmake_dependent_option
# will ensure that shared library support will be disabled if the system does
# not support it.

# Setup shared library / -fPIC stuff
get_property(SHARED_LIBS_SUPPORTED GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS)
Expand All @@ -78,8 +91,6 @@ set_property(CACHE ZFP_BIT_STREAM_WORD_SIZE PROPERTY STRINGS "8;16;32;64")
option(ZFP_WITH_BIT_STREAM_STRIDED
"Enable strided access for progressive zfp streams" OFF)

option(ZFP_WITH_COMPRESSION "Enable or disable compression" ON)

option(ZFP_WITH_ALIGNED_ALLOC "Enabled aligned memory allocation" OFF)

option(ZFP_WITH_CACHE_TWOWAY "Use two-way skew-associative cache" OFF)
Expand Down Expand Up @@ -114,5 +125,13 @@ endif()
# Header install
#------------------------------------------------------------------------------#

install(DIRECTORY inc/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
install(DIRECTORY array/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
install(DIRECTORY array/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})

#------------------------------------------------------------------------------#
# Build type: one of None, Debug, Release, RelWithDebInfo, MinSizeRel
#------------------------------------------------------------------------------#

if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Type of CMake build" FORCE)
endif()
30 changes: 6 additions & 24 deletions Config
Original file line number Diff line number Diff line change
@@ -1,38 +1,23 @@
# compiler settings -----------------------------------------------------------
# default compiler settings ---------------------------------------------------

# GNU compiler
CC = gcc
CXX = g++
FLAGS = -O3 -fPIC -Wall -I../inc $(DEFS)
CFLAGS = $(FLAGS) -std=c89 -Wno-unused-function
#CFLAGS = $(FLAGS) -std=c99
CXXFLAGS = $(FLAGS) -std=c++98

# IBM compiler
# CC = xlc
# CXX = xlc++
# CFLAGS = -O2 -qmaxmem=-1 -qpic=large -I../inc $(DEFS)
# CXXFLAGS= $(CFLAGS)
FLAGS = -O3 -fPIC -Wall -I../include $(DEFS)
# CFLAGS = -std=c89 -Wno-unused-function $(FLAGS)
CFLAGS = -std=c99 $(FLAGS)
CXXFLAGS = -std=c++98 $(FLAGS)

# optional compiler macros ----------------------------------------------------

# use smaller bit stream word type for finer rate granularity
# DEFS += -DBIT_STREAM_WORD_TYPE=uint8
# DEFS += -DBIT_STREAM_WORD_TYPE=uint16
# DEFS += -DBIT_STREAM_WORD_TYPE=uint32
# DEFS += -DBIT_STREAM_WORD_TYPE=uint64

# enable strided access for progressive zfp streams
# DEFS += -DBIT_STREAM_STRIDED

# run regression test with medium sized arrays
# DEFS += -DTESTZFP_MEDIUM_ARRAYS

# run regression test with large arrays
# DEFS += -DTESTZFP_LARGE_ARRAYS

# run diffusion without compression
# DEFS += -DWITHOUT_COMPRESSION

# use aligned memory allocation
# DEFS += -DALIGNED_ALLOC

Expand All @@ -44,6 +29,3 @@ CXXFLAGS = $(FLAGS) -std=c++98

# count cache misses
# DEFS += -DCACHE_PROFILE

# for recent versions of MSVC
# DEFS += -DHAVE_C99_MATH
Loading

0 comments on commit bf7c8be

Please sign in to comment.