Please, help us to better serve our user community by answering the following short survey: https://www.hdfgroup.org/website-survey/
HDF5  1.14.4.3
API Reference
 
Loading...
Searching...
No Matches
Writing by Chunk

Navigate back: Main / Getting Started with HDF5 / A Brief Introduction to Parallel HDF5


In this example each process writes a "chunk" of data to a dataset. The C and Fortran 90 examples result in the same data layout in the file.

Figure a C Example Figure b Fortran Example

For this example, four processes are used, and a 4 x 2 chunk is written to the dataset by each process.

To do this, you would:

  • Use the block parameter to specify a chunk of size 4 x 2 (or 2 x 4 for Fortran).
  • Use a different offset (start) for each process, based on the chunk size:
    CProcess 0Process 1Process 2Process 3
    offset[0] = 0offset[0] = 0offset[0] = 4offset[0] = 4
    offset[1] = 0offset[1] = 2offset[1] = 0offset[1] = 2
    FortranProcess 0Process 1Process 2Process 3
    offset(1) = 0offset(1) = 2offset(1) = 0offset(1) = 2
    offset(2) = 0offset(2) = 0offset(2) = 4offset(2) = 4

For example, the offset and block parameters for Process 2 would look like:

Figure a C Example Figure b Fortran Example

Below are example programs for writing hyperslabs by pattern in Parallel HDF5:

hyperslab_by_chunk.c
hyperslab_by_chunk.F90

The following is the output from h5dump for the HDF5 file created in this example:

HDF5 "SDS_chnk.h5" {
GROUP "/" {
DATASET "IntArray" {
DATATYPE H5T_STD_I32BE
DATASPACE SIMPLE { ( 8, 4 ) / ( 8, 4 ) }
DATA {
1, 1, 2, 2,
1, 1, 2, 2,
1, 1, 2, 2,
1, 1, 2, 2,
3, 3, 4, 4,
3, 3, 4, 4,
3, 3, 4, 4,
3, 3, 4, 4
}
}
}
}
#define H5T_STD_I32BE
Definition H5Tpublic.h:317

The h5dump utility is written in C so the output is in C order.


Navigate back: Main / Getting Started with HDF5 / A Brief Introduction to Parallel HDF5