Please, help us to better serve our user community by answering the following short survey: https://www.hdfgroup.org/website-survey/
HDF5  1.14.4.3
API Reference
 
Loading...
Searching...
No Matches
Writing by Contiguous Hyperslab

Navigate back: Main / Getting Started with HDF5 / A Brief Introduction to Parallel HDF5


This example shows how to write a contiguous buffer in memory to a contiguous hyperslab in a file. In this case, each parallel process writes a contiguous hyperslab to the file.

In the C example (figure a), each hyperslab in memory consists of an equal number of consecutive rows. In the FORTRAN 90 example (figure b), each hyperslab in memory consists of an equal number of consecutive columns. This reflects the difference in the storage order for C and FORTRAN 90.

Figure a C Example Figure b Fortran Example

Writing a Contiguous Hyperslab in C

In this example, you have a dataset of 8 (rows) x 5 (columns) and each process writes an equal number of rows to the dataset. The dataset hyperslab is defined as follows:

count [0] = dimsf [0] / number_processes
count [1] = dimsf [1]

where,

dimsf [0] is the number of rows in the dataset
dimsf [1] is the number of columns in the dataset

The offset for the hyperslab is different for each process:

offset [0] = k * count[0]
offset [1] = 0

where,

"k" is the process id number
count [0] is the number of rows written in each hyperslab
offset [1] = 0 indicates to start at the beginning of the row

The number of processes that you could use would be 1, 2, 4, or 8. The number of rows that would be written by each slab is as follows:

Processes Size of count0
18
24
42
81

If using 4 processes, then process 1 would look like:

The code would look like the following:

71 /*
72 * Each process defines dataset in memory and writes it to the hyperslab
73 * in the file.
74 */
75 count[0] = dimsf[0]/mpi_size;
76 count[1] = dimsf[1];
77 offset[0] = mpi_rank * count[0];
78 offset[1] = 0;
79 memspace = H5Screate_simple(RANK, count, NULL);
80
81 /*
82 * Select hyperslab in the file.
83 */
84 filespace = H5Dget_space(dset_id);
85 H5Sselect_hyperslab(filespace, H5S_SELECT_SET, offset, NULL, count, NULL);
@ H5S_SELECT_SET
Definition H5Spublic.h:87
hid_t H5Dget_space(hid_t dset_id)
Returns an identifier for a copy of the dataspace for a dataset.
herr_t H5Sselect_hyperslab(hid_t space_id, H5S_seloper_t op, const hsize_t start[], const hsize_t stride[], const hsize_t count[], const hsize_t block[])
Selects a hyperslab region to add to the current selected region.
hid_t H5Screate_simple(int rank, const hsize_t dims[], const hsize_t maxdims[])
Creates a new simple dataspace and opens it for access.

Below is the example program:

hyperslab_by_row.c

If using this example with 4 processes, then,

  • Process 0 writes "10"s to the file.
  • Process 1 writes "11"s.
  • Process 2 writes "12"s.
  • Process 3 writes "13"s.

The following is the output from h5dump for the HDF5 file created by this example using 4 processes:

HDF5 "SDS_row.h5" {
GROUP "/" {
DATASET "IntArray" {
DATATYPE H5T_STD_I32BE
DATASPACE SIMPLE { ( 8, 5 ) / ( 8, 5 ) }
DATA {
10, 10, 10, 10, 10,
10, 10, 10, 10, 10,
11, 11, 11, 11, 11,
11, 11, 11, 11, 11,
12, 12, 12, 12, 12,
12, 12, 12, 12, 12,
13, 13, 13, 13, 13,
13, 13, 13, 13, 13
}
}
}
}
#define H5T_STD_I32BE
Definition H5Tpublic.h:317

Writing a Contiguous Hyperslab in Fortran

In this example you have a dataset of 5 (rows) x 8 (columns). Since a contiguous hyperslab in Fortran 90 consists of consecutive columns, each process will be writing an equal number of columns to the dataset.

You would define the size of the hyperslab to write to the dataset as follows:

count(1) = dimsf(1)
count(2) = dimsf(2) / number_of_processes

where,

dimsf(1) is the number of rows in the dataset
dimsf(2) is the number of columns

The offset for the hyperslab dimension would be different for each process:

offset (1) = 0
offset (2) = k * count (2)

where,

offset (1) = 0 indicates to start at the beginning of the column
"k" is the process id number
"count(2) is the number of columns to be written by each hyperslab

The number of processes that could be used in this example are 1, 2, 4, or 8. The number of columns that could be written by each slab is as follows:

Processes Size of count (2)(# of columns)
18
24
42
81

If using 4 processes, the offset and count parameters for Process 1 would look like:

The code would look like the following:

69 ! Each process defines dataset in memory and writes it to the hyperslab
70 ! in the file.
71 !
72 count(1) = dimsf(1)
73 count(2) = dimsf(2)/mpi_size
74 offset(1) = 0
75 offset(2) = mpi_rank * count(2)
76 CALL h5screate_simple_f(rank, count, memspace, error)
77 !
78 ! Select hyperslab in the file.
79 !
80 CALL h5dget_space_f(dset_id, filespace, error)
81 CALL h5sselect_hyperslab_f (filespace, H5S_SELECT_SET_F, offset, count, error)

Below is the F90 example program which illustrates how to write contiguous hyperslabs by column in Parallel HDF5:

hyperslab_by_col.F90

If you run this program with 4 processes and look at the output with h5dump you will notice that the output is much like the output shown above for the C example. This is because h5dump is written in C. The data would be displayed in columns if it was printed using Fortran 90 code.


Navigate back: Main / Getting Started with HDF5 / A Brief Introduction to Parallel HDF5