Blobs

Motivation

A few application developers have requested the capability to store an arbitrarily-sized object in an exodus file. The size of the object is not related to any of the existing exodus entities -- nodes, elements, sides, edges, or faces. An example of a blob would be an arbitrary vector of floating-point values; the coefficients of a solution matrix.

The blob concept would allow one or more arbitrarily sized objects to be defined. The object would have:

Size (Is 1D OK, or should it support 2D a[rows][cols])
- Supports 1D only.
The type would correspond to the native floating point type specified for the exodus file (float or double)
Unique id (required, positive, unique among blobs)
Unique name (required, unique among all blobs/blocks/sets/assemblies)
Zero or more named attributes.
- Attribute is per blob, not a value for each entry in the blob
- attribute can be of type -- integer, char* double
- if type is integer or double, then there can be multiple values per attribute.
Transient data -- zero or more named floating point values which are output on each timestep.
- Value for each item in the blob for each variable
- Names should be unique across all transient reduction variable names in the model.
Transient Reduction Data -- zero or more named floating point values which are output on each timestep.
- Single value per timestep per blob for each variable
- Names should be unique across all transient reduction variable names in the model.
Zero or more "named maps" can be specified
- The map will have a size equal to the size of the blob
- The map entries will be 32- or 64-bit integers (based on exodus integer size)
- The value of the entries in the map(s) is arbitrary and it is the responsibility of the writer/reader to agree on the map conventions and the meaning of the values.

Parallel Considerations

Client will be responsible for handling all parallel issues.
Assume that blobs will only be used in the "1->N->1" mode -- parallel application reads and writes from/to a single file on a parallel filesystem.
Will need to provide partial read/write functions for variables and maps so client can efficiently read/write blob data in a parallel execution.

C API

Determine number of blobs on an exodus file:

 int num_blob = ex_inquire_int(exoid, EX_INQ_BLOB);

Define number of blobs which will be defined in an exodus file:

The number of blobs does not need to be specified prior to defining an blob.

Get / Put names of all blobs or an individual blob:

Blob names are set and queried via the ex_put_blobs and ex_get_blobs functions. They are somewhat different from other object types in that the names are not defined via the ex_put_name calls. However, the names can be queried using ex_get_name() although it is recommended to use the ex_get_blob() function.

 ex_get_name(exoid, EX_BLOB, blob_id, name);
 ex_get_names(exoid, EX_BLOB,  names[]);

Define and output one or more blob(s)

typedef struct ex_blob
{
  int64_t        id;
  char           *name;
  int64_t        size;
} ex_blob;

ex_put_blob(exoid, ex_blob blob);
ex_put_blobs(exoid, num_blob, ex_blob *blobs);

Defines and outputs an blob with id id and name name of size size entries.

NOTE: Efficiency -- it is more efficient to define multiple blobs via a call to ex_put_blobs() then to call ex_put_blob() multiple times.

   /* conceptual; not real C code */
   ex_blob blob[3];
   blob.id = {11, 22, 3};
   blob.name = {"Fred", "Wilma", "BamBam"};
   blob.size = {200000, 100000, 2000000000};
   ex_put_blobs(exoid, 3, blob);

Read all blobs

   int num_blob = ex_inquire_int(exoid, EX_INQ_BLOB);
   int64_t ids[num_blob];
   ex_get_ids(exoid, EX_BLOB, ids);

   int name_len = ex_inquire_int(exoid, EX_INQ_DB_MAX_USED_NAME_LENGTH);
   struct ex_blob blobs[num_blob];
   for (int i=0; i < num_blob; i++) {
      blobs[i].name = malloc(name_len + 1);
   }

   /* Get parameters for all blobs */
   ex_get_blobs(exoid, blobs);

Define/Query blob attributes and attribute names

Described in https://github.com/gsjaardema/seacas/wiki/Entity-Attributes

An blob attribute is similar to an IOSS property consisting of a name, a type, and a value or values. It is not a value per entity in the blob, but a value for the blob. For now, they types will be limited to text, integer, and double to provide capability without the complexity of supporting the many types available in NetCDF-4 including user-defined types. Note that an attribute can have multiple values, for example if the attribute is a range, a single attributed could have two double values -- {1.0, 100.0}.

Define/Query maps and map data

[ ] TODO: Add api functions

Define/Query number of transient variables on an blob

 ex_put_variable_param(exoid, EX_BLOB, num_variables);
 ex_get_variable_param(exoid, EX_BLOB, &num_variables);

[UNDECIDED] Can also define and query the blob truth table which will specify which variables are defined on which blobs.

Define/Query number of transient reduction variables on an blob

 ex_put_reduction_variable_param(exoid, EX_BLOB, num_variables);
 ex_get_reduction_variable_param(exoid, EX_BLOB, &num_variables);

Note that there will be a value written to and read from the database for all variables on all blobs, but only the values corresponding to a defined variable will be valid values.

Define / Query names of variables on an blob

 ex_put_variable_names(exoid, EX_BLOB, num_variables, names);
 ex_get_variable_names(exoid, EX_BLOB, num_variables, names);
 ex_put_variable_name(exoid, EX_BLOB, index, name);
 ex_get_variable_name(exoid, EX_BLOB, index, name);

Define / Query names of reduction variables on an blob

 ex_put_reduction_variable_names(exoid, EX_BLOB, num_variables, names);
 ex_get_reduction_variable_names(exoid, EX_BLOB, num_variables, names);
 ex_put_reduction_variable_name(exoid, EX_BLOB, index, name);
 ex_get_reduction_variable_name(exoid, EX_BLOB, index, name);

Write / Read data for blob variable(s) at a time step

All variables for a single blob (with id id):

  ex_get_vars(exoid, step, EX_BLOB, id, num_assem_vars, values);
  ex_put_vars(exoid, step, EX_BLOB, id, num_assem_vars, values);

[x] Need partial get/put functions.

Write / Read data for blob reduction variable(s) at a time step