Skip to content

Commit

Permalink
Add missing files
Browse files Browse the repository at this point in the history
  • Loading branch information
gsjaardema committed Nov 5, 2020
1 parent dbed800 commit 9a7ebaa
Show file tree
Hide file tree
Showing 49 changed files with 46,291 additions and 0 deletions.
163 changes: 163 additions & 0 deletions 64-bit-integer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
## Integer Bulkdata Storage Details

The EXODUS database can store integer bulk data, entity map data, and
mesh entity (block/set) ids in either 32-bit or 64-bit integer format. The data
considered "bulk data" are:

* element, face, and edge connectivity lists,
* element, face, edge, and node set entity lists,

The entity map data is any data stored in one of the 'map' objects on
the exodus file. This includes:

* id maps
* number maps
* order maps
* processor node maps
* processor element maps.

A mesh entity id is the id of any block (element block, edge block,
...); set (node set, face set, ...), coordinate frame, and
communication map.


When an EXODUS file is created via the `ex_create()` function, the
'mode' argument provides the method for specifying how integer data
will be passed as arguments to the API functions and also how the
integer data will be stored on the database. The `ex_open()` function
provides the same method for specifying how integer data will be
passed as arguments to the API functions.

The method uses the 'mode' argument to the `ex_open()` and
`ex_create()` functions. The mode is a 32-bit integer in which certain
bits are turned on by or'ing certain predefined constants.

```
exoid = ex_create( EX_TEST_FILENAME,
EX_CLOBBER|EX_MAPS_INT64_DB|EX_MAPS_INT64_API,
&appWordSize, &diskWordSize );
```

The constants related to the integer size (32-bit or 64-bit) specification are:

Constant | Description
-------------------|--------------
`EX_MAPS_INT64_DB` | entity map data
`EX_IDS_INT64_DB` | mesh entity ids
`EX_BULK_INT64_DB` | bulk data
`EX_ALL_INT64_DB` | (the above 3 or'd together)
`EX_MAPS_INT64_API` | entity map data
`EX_IDS_INT64_API` | mesh entity ids
`EX_BULK_INT64_API` | bulk data
`EX_ALL_INT64_API` | (the above 3 or'd together)

The constants that end with `_DB` specify that that particular integer
data is stored on the database as 64-bit integers; the constants that
end with `_API` specify that that particular integer data is passed
to/from API functions as 64-bit integers.

If the range of the data being transmitted is larger than the
permitted integer range (for example, if the data is stored on the
database truly requires 64-bit ints and the application specifies
passing data as 32-bit ints), the api function will return an error.

The three types of integer data whose storage can be specified are:

* maps (`EX_MAPS_INT64_`),
* "bulk data" including connectivity lists and entity lists
(`EX_BULK_INT64_`), and
* entity ids which are the ids of element, face, edge, and node
sets and blocks; and map ids (`EX_IDS_INT64_`)

The function `ex_int64_status(exoid)` is used to determine the integer
storage types being used for the EXODUS database 'exoid'. The function
returns an integer which can be and'ed with the above flags to
determine either the storage type or function parameter type.

For example, if

```
(EX_MAPS_INT64_DB & ex_int64_status(exoid))
```
is true, then map data is being stored as 64-bit integers for that database.

It is not possible to determine the integer data size on a database
without opening the database via an `ex_open()` call. However, the
integer size specification for API functions can be changed at any
time via the `ex_set_int64_status(exoid, mode)` function. The mode is
one or more of `EX_MAPS_INT64_API`, `EX_IDS_INT64_API`, or
`EX_BULK_INT64_API`, or'd together. Any exodus function calls after
that point will use the specified integer size. Note that a call to
`ex_set_int64_status(exoid, mode)` overrides any previous setting for
the integer sizes used in the API. The `ex_create()` function is the
only way to specify the size of the integers that will be written to a database.


## IOSS: IO System libraries

The IO System (Ioss) is also 64-bit integer capable. When Ioss opens
and reads the intial metadata from an exodus file, it will determine
the size of integers stored on the database. If either 'maps' or 'bulk
data' are stored as 64-bit integers, then Ioss will automatically
specify that all integers will be transferred as 64-bit integers.
Entity (block/set) ids are always treated as 64-bit integers in Ioss.

The setting can be queried via the *Ioss::DatabaseIO*
`int_byte_size_api()` member function call which will return either 4 or
8. The setting can be modified via the `set_int_byte_size_api(mode)`
function call where mode is one of:

* `Ioss::USE_INT64_API` or
* `Ioss::USE_INT32_API`.

The integer byte size setting should only be changed prior to assiging
the database to an *Ioss::Region* since fields are created using the
setting when the meta data of the database is read. An *Ioss::Field*
that stores 64-bit integers will have a type of `Ioss::Field::INT64`
and a field that stores 32-bit integers will have a type of
`Ioss::Field::INTEGER` (`Ioss::Field::INT32` can also be used).

The Ioss io_shell application will maintain the integer size of the
input database to the output database by default. If the `--64`
option is specified, then the output database will use 64-bit integers
for all quantities.


## SEACAS Applications

Many of the SEACAS applications have been modified to be 64-bit
capable. They include:

* conjoin
* ejoin
* epu
* exodiff
* nem_spread
* nem_slice -- Note that only the linear, random, or scattered
decomposition options are allowed for 64-bit integer
meshes. Other options are being investigated for the
slicing of large meshes.

* algebra, blot, exotxt, gen3d, genshell, gjoin, grepos, explore,
mapvar, numbers, txtexo

## AVAILABILITY:

The 64-bit integer capability is currently available in the following
locations:

* SEACAS repository https://github.com/gsjaardema/seacas.git
* Sierra repository (Internal)
* Trilinos repository https://github.com/trilinos/Trilinos

## Other Details:

* Exodus requires netcdf-4.1.2 or later; netcdf-4.3.3.1 or later is recommended

* The Nemesis library has been absorbed into the exodus library.
* All api functions that were `ne_*` are now `ex_*`
* A wrapper exists in the old nemesis library that will forward all
`ne_*` calls to the corresponding `ex_*` call to maintain backward
compatability with unconverted applications.
* In other words, we recommend using the exodus library for nemesis
routines, but the nemesis library can be used if you want.
168 changes: 168 additions & 0 deletions 64-bit-integer.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
Integer Bulkdata Storage Details

The EXODUS database can store integer bulk data, entity map data, and
mesh entity (block/set) ids in either 32-bit or 64-bit integer format. The data
considered "bulk data" are:

* element, face, and edge connectivity lists,
* element, face, edge, and node set entity lists,

The entity map data is any data stored in one of the 'map' objects on
the exodus file. This includes:
* id maps
* number maps
* order maps
* processor node maps
* processor element maps.

A mesh entity id is the id of any block (element block, edge block,
...); set (node set, face set, ...), coordinate frame, and
communication map.


When an EXODUS file is created via the ex_create() function, the
'mode' argument provides the method for specifying how integer data
will be passed as arguments to the API functions and also how the
integer data will be stored on the database. The ex_open() function
provides the same method for specifying how integer data will be
passed as arguments to the API functions.

The method uses the 'mode' argument to the ex_open() and
ex_create() functions. The mode is a 32-bit integer in which certain
bits are turned on by or'ing certain predefined constants.

exoid = ex_create( EX_TEST_FILENAME,
EX_CLOBBER|EX_MAPS_INT64_DB|EX_MAPS_INT64_API,
&appWordSize, &diskWordSize );

The constants related to the integer size (32-bit or 64-bit) specification are:

EX_MAPS_INT64_DB -- entity map data
EX_IDS_INT64_DB -- mesh entity ids
EX_BULK_INT64_DB -- bulk data
EX_ALL_INT64_DB -- (the above 3 or'd together)
EX_MAPS_INT64_API -- entity map data
EX_IDS_INT64_API -- mesh entity ids
EX_BULK_INT64_API -- bulk data
EX_ALL_INT64_API -- (the above 3 or'd together)

The constants that end with "_DB" specify that that particular integer
data is stored on the database as 64-bit integers; the constants that
end with "_API" specify that that particular integer data is passed
to/from API functions as 64-bit integers.

If the range of the data being transmitted is larger than the
permitted integer range (for example, if the data is stored on the
database truly requires 64-bit ints and the application specifies
passing data as 32-bit ints), the api function will return an error.

The three types of integer data whose storage can be specified are:

* maps (EX_MAPS_INT64_),
* "bulk data" including connectivity lists and entity lists
(EX_BULK_INT64_), and
* entity ids which are the ids of element, face, edge, and node
sets and blocks; and map ids (EX_IDS_INT64_)

The function ex_int64_status(exoid) is used to determine the integer
storage types being used for the EXODUS database 'exoid'. The function
returns an integer which can be and'ed with the above flags to
determine either the storage type or function parameter type.

For example, if (EX_MAPS_INT64_DB & ex_int64_status(exoid)) is true,
then map data is being stored as 64-bit integers for that database.

It is not possible to determine the integer data size on a database
without opening the database via an ex_open() call. However, the
integer size specification for API functions can be changed at any
time via the ex_set_int64_status(exoid, mode) function. The mode is
one or more of EX_MAPS_INT64_API, EX_IDS_INT64_API, or
EX_BULK_INT64_API, or'd together. Any exodus function calls after
that point will use the specified integer size. Note that a call to
ex_set_int64_status(exoid, mode) overrides any previous setting for
the integer sizes used in the API. The ex_create() function is the
only way to specify the integer sizes specification for database
integers.

========================================================================

IOSS: IO System libraries

The IO System (Ioss) is also 64-bit integer capable. When Ioss opens
and reads the intial metadata from an exodus file, it will determine
the size of integers stored on the database. If either 'maps' or 'bulk
data' are stored as 64-bit integers, then Ioss will automatically
specify that all integers will be transferred as 64-bit integers.
Entity (block/set) ids are always treated as 64-bit integers in Ioss.

The setting can be queried via the Ioss::DatabaseIO
int_byte_size_api() member function call which will return either 4 or
8. The setting can be modified via the set_int_byte_size_api(mode)
function call where mode is one of:

* Ioss::USE_INT64_API or
* Ioss::USE_INT32_API.

The integer byte size setting should only be changed prior to assiging
the database to an Ioss::Region since fields are created using the
setting when the meta data of the database is read. An Ioss::Field
that stores 64-bit integers will have a type of 'Ioss::Field::INT64'
and a field that stores 32-bit integers will have a type of
'Ioss::Field::INTEGER' (Ioss::Field::INT32 can also be used).

The Ioss io_shell application will maintain the integer size of the
input database to the output database by default. If the '--64'
option is specified, the the output database will use 64-bit integers
for all quantities.

========================================================================

SEACAS Applications

Many of the SEACAS applications have been modified to be 64-bit
capable. The include:

* epu
* nem_spread
* nem_slice -- Note that only the linear, random, or scattered
decomposition options are allowed for 64-bit integer
meshes. Other options are being investigated for the
slicing of large meshes.

* algebra, blot, exotxt, gen3d, genshell, gjoin, grepos, explore,
mapvar, numbers, txtexo
-- These will probably work, but have not all been tested on
very large models. Blot has read and displayed a 4.4 billion
element model; explore and grepos have also read/written the
same model, but that is the extent of the testing.

Still need conversion:
* exodiff
* conjoin
* ejoin


========================================================================
AVAILABILITY:

The 64-bit integer capability is currently available in the following
locations:

* SEACAS repository
* Sierra repository
* Sourceforge exodusII repository
* Sourceforge seacas repository

The Trilinos repository will be updated as soon as a newer version of
netcdf is available on the Trilinos build/test machines.

========================================================================
Other Details:

* Exodus requires netcdf-4.1.2 or later.

* The Nemesis library has been absorbed into the exodus library.
-- All api functions that were ne_* are now ex_*
-- A wrapper exists in the old nemesis library that will forward all
ne_* calls to the corresponding ex_* call to maintain backward
compatability with unconverted applications.
36 changes: 36 additions & 0 deletions Exodus-Long-Names.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
On write, you can call:

```
ex_set_option(exoid, EX_OPT_MAX_NAME_LENGTH, {max_name_length});
```

or equivalently

```
ex_set_max_name_length(exoid, {max_name_length});
```

which tells the database that you will be using names of that length or shorter.

This same call on read will tell the database to truncate names at
that length. If not called, then the `{max_name_length}` is assumed to
be 32 characters which has been the historic maximum name length for
exodus.

Following this call, you can define (i.e., read/write) names of any
size; if the names are longer than `{max_name_length}`, then they will be truncated otherwise they will pass through unchanged.

There are three queries that can be made to `ex_inquire` or
`ex_inquire_int`:

* `EX_INQ_DB_MAX_ALLOWED_NAME_LENGTH` -- returns the value of the
maximum size that can be specified for `max_name_length`
(netcdf/hdf5 limitation)
* `EX_INQ_DB_MAX_USED_NAME_LENGTH` -- returns the size of the longest
name on the database.
* `EX_INQ_MAX_READ_NAME_LENGTH` -- returns the maximum name length
size that will be passed back to the client. 32 by default,
set by the previously mentioned `ex_set_option()` or
`ex_set_max_name_length()` call.

The length of the QA records is not affected by this setting and each entry in the QA record is still limited to 32 characters.
Binary file added ExodusII-Addendum.pdf
Binary file not shown.
Loading

0 comments on commit 9a7ebaa

Please sign in to comment.