Command-Line Tools for Converting

Contents:

• Output HDF5 Dataset into an ASCII File (to Import into Excel and Other Applications)
• Output HDF5 Dataset into Binary File
• Export from h5dump and Import into HDF5

Output HDF5 Dataset into an ASCII File (to Import into Excel and Other Applications)

The h5dump utility can be used to convert an HDF5 dataset into an ASCII file, which can then be imported into Excel and other applications. The following options are used:

 -d D, --dataset=D

 -o F, --output=F 

 -y, --noindex

 -w N, --width=N

As an example, h5_crtdat.c from the Creating a Dataset HDF5 Tutorial topic, creates the file dset.h5 with a dataset  /dset  that is a 4 x 6 integer array. The following is displayed when viewing dset.h5 with h5dump:

$ h5dump dset.h5
HDF5 "dset.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE  H5T_STD_I32BE
      DATASPACE  SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
      DATA {
      (0,0): 1, 2, 3, 4, 5, 6,
      (1,0): 7, 8, 9, 10, 11, 12,
      (2,0): 13, 14, 15, 16, 17, 18,
      (3,0): 19, 20, 21, 22, 23, 24
      }
   }
}
} 

The following command will output the values of the  /dset  dataset to the ASCII file dset.asci:

h5dump -d /dset -o dset.asci -y -w 50 dset.h5

In particular, note that:

• The default behavior of h5dump is to print indices, and the -y option suppresses this.

• The -w 50 option tells h5dump to allow 50 columns for outputting the data. The value specified must be large enough to accomodate the dimension size of the dataset multiplied by the number of positions and spaces needed to print each value. If the value is not large enough, the output will wrap to the next line, and the data will not display as expected in Excel or other applications. To ensure that the output does not wrap to the next line, you can also specify 0 (zero) for the -w option.

In addition to creating the ASCII file dset.asci, the above command outputs the metadata of the specified dataset:

HDF5 "dset.h5" {
DATASET "/dset" {
   DATATYPE  H5T_STD_I32BE
   DATASPACE  SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
   DATA {
   }
}
}

The dset.asci file will contain the values for the dataset:

      1, 2, 3, 4, 5, 6,
      7, 8, 9, 10, 11, 12,
      13, 14, 15, 16, 17, 18,
      19, 20, 21, 22, 23, 24

Output HDF5 Dataset into Binary File

The h5dump utility can be used to convert an HDF5 dataset to a binary file with the following options:

 -d D, --dataset=D

 -o F, --output=F 

 -b B, --binary=B

As an example, h5_crtdat.c from the Creating a Dataset HDF5 Tutorial topic, creates the file dset.h5 with a dataset  /dset  that is a 4 x 6 integer array. The following is displayed when viewing dset.h5 with h5dump:

$ h5dump -d /dset/ dset.h5
HDF5 "dset.h5" {
DATASET "/dset/" {
   DATATYPE  H5T_STD_I32BE
   DATASPACE  SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
   DATA {
   (0,0): 1, 2, 3, 4, 5, 6,
   (1,0): 7, 8, 9, 10, 11, 12,
   (2,0): 13, 14, 15, 16, 17, 18,
   (3,0): 19, 20, 21, 22, 23, 24
   }
}
}

As specified by the -d and -o options, the following h5dump command will output the values of the dataset  /dset  to a file called dset.bin. The -b option specifies that the output will be binary in Little Endian format (LE).

h5dump -d /dset -b LE -o dset.bin dset.h5

This command outputs the metadata for the dataset, as well as creating the binary file dset.bin:

HDF5 "dset.h5" {
DATASET "/dset" {
   DATATYPE  H5T_STD_I32BE
   DATASPACE  SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
   DATA {
   }
}
}

If you look at the resulting dset.bin file with a binary editor, you will see that it contains the dataset's values. For example (on Linux) you will see:

$ od -t d dset.bin
0000000           1           2           3           4
0000020           5           6           7           8
0000040           9          10          11          12
0000060          13          14          15          16
0000100          17          18          19          20
0000120          21          22          23          24
0000140

Export from h5dump and Import into HDF5

The h5import utility can use the output of h5dump as input to create a dataset or file.

The h5dump utility must first create two files:

• A DDL file, which will be used as an h5import configuration file
• A raw data file containing the data to be imported

The DDL file must be generated with the h5dump -p option, to generate properties.

The raw data file that can be imported into HDF5 using this method may contain either numeric or string data with the following restrictions:

• Numeric data requires the use of the h5dump -b option to produce a binary data file.
• String data must be written with the h5dump -y and --width=1 options, generating a single column of strings without indices.

Two examples follow: the first imports a dataset with a numeric datatype. Note that numeric data requires the use of the h5dump -b option to produce a binary data file. The example program (h5_crtdat.c) that creates this file is included with the HDF5 Introductory tutorial and can be obtained from the HDF5 Introductory Examples page:

h5dump -p -d "/dset" --ddl=dsetbin.dmp -o dset.bin -b dset.h5
h5import dset.bin -c dsetbin.dmp -o new-dset.h5

The output before and after running these commands is shown below:

$  h5dump dset.h5
HDF5 "dset.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE  H5T_STD_I32BE
      DATASPACE  SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
      DATA {
      (0,0): 1, 2, 3, 4, 5, 6,
      (1,0): 7, 8, 9, 10, 11, 12,
      (2,0): 13, 14, 15, 16, 17, 18,
      (3,0): 19, 20, 21, 22, 23, 24
      }
   }
}
}
$ h5dump -p -d "/dset" --ddl=dsetbin.dmp -o dset.bin -b dset.h5

$ h5import dset.bin -c dsetbin.dmp -o new-dset.h5

$ h5dump new-dset.h5
HDF5 "new-dset.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE  H5T_STD_I32BE
      DATASPACE  SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
      DATA {
      (0,0): 1, 2, 3, 4, 5, 6,
      (1,0): 7, 8, 9, 10, 11, 12,
      (2,0): 13, 14, 15, 16, 17, 18,
      (3,0): 19, 20, 21, 22, 23, 24
      }
   }
}
}

The second example imports string data. The example program that creates this file can be downloaded from the C Examples by API page.

Note that string data requires use of the h5dump -y option to exclude indexes and the h5dump --width=1 option to generate a single column of strings. The -o option outputs the data into an ASCII file.

h5dump -p -d "/DS1" -O vlstring.dmp -o vlstring.ascii -y --width=1 h5ex_t_vlstring.h5
h5import vlstring.ascii -c vlstring.dmp -o new-vlstring.h5

The output before and after running these commands is shown below:

$ h5dump h5ex_t_vlstring.h5
HDF5 "h5ex_t_vlstring.h5" {
GROUP "/" {
   DATASET "DS1" {
      DATATYPE  H5T_STRING {
         STRSIZE H5T_VARIABLE;
         STRPAD H5T_STR_SPACEPAD;
         CSET H5T_CSET_ASCII;
         CTYPE H5T_C_S1;
      }
      DATASPACE  SIMPLE { ( 4 ) / ( 4 ) }
      DATA {
      (0): "Parting", "is such", "sweet", "sorrow."
      }
   }
}
}

$ h5dump -p -d "/DS1" -O vlstring.dmp -o vlstring.ascii -y --width=1 h5ex_t_vlstring.h5

$ h5import vlstring.ascii -c vlstring.dmp -o new-vlstring.h5

$ h5dump new-vlstring.h5
HDF5 "new-vlstring.h5" {
GROUP "/" {
   DATASET "DS1" {
      DATATYPE  H5T_STRING {
         STRSIZE H5T_VARIABLE;
         STRPAD H5T_STR_NULLTERM;
         CSET H5T_CSET_ASCII;
         CTYPE H5T_C_S1;
      }
      DATASPACE  SIMPLE { ( 4 ) / ( 4 ) }
      DATA {
      (0): "Parting", "is such", "sweet", "sorrow."
      }
   }
}
}