This indicates that the HDF5 library was being built with the wrong C compiler that does not support MPI. Follow the instructions in the
./release_docs/INSTALL_parallel file file, under the "IBM SP" section. Though it is geared towards a particular IBM installation, it does applies to the Regatta. The only exception is that you probably don't do the:
setenv LLNL_COMPILE_SINGLE_THREADED TRUE
On the other hand, IBM has hundreds of environment variables and various compilers. Each site has various individual settings. Therefore, you should first try to compile and run some simple MPI-IO programs, both C and Fortran90, with the compilers,
mpxlf_r, respectively. Once you get your MPI parallel environment set up, you can proceed with the instructions mentioned above.