Page tree

Members of the HDF5 users community can register VOL connectors for use with HDF5.

To register a VOL connector please contact The HDF Helpdesk with the following information: 

  • Contact information for the developer requesting a new identifier
  • Short description of the new connector
  • Links to any relevant information including licensing information 

Here is the current policy regarding VOL connector identifier assignment:

Valid VOL connector identifiers can have values from 0 through 255 for connectors defined by the HDF5 library. Values 256 through 511 are available for testing new VOL connectors.  Subsequent values should be obtained by contacting the The HDF Help Desk.

Please contact the maintainer of a VOL connector for help implementing the plugin.

List of VOL Connectors Registered with The HDF Group
ConnectorConnector IdentifierSearch Name*Short DescriptionURL

Contacts

Asynchronous I/O512asyncProvides support for asynchronous operations to HDF5https://github.com/hpc-io/vol-async

 Suren Byna (sbyna at lbl dot gov)

Cache513cacheProvides support for multi-level, multi-location data caching to dataset I/O operationshttps://github.com/hpc-io/vol-cache

Suren Byna (sbyna at lbl dot gov)

Log-based514LOG

The log-based VOL plugin stores HDF5 datasets in a log-based storage layout.

In this layout, data of multiple write requests made by an MPI process are appended one after another in the file. Such I/O strategy can avoid the expensive inter-process communication and I/O serialization due to file lock contentions when storing data in the canonical order. Through the log-based VOL, existing HDF5 programs can achieve a better parallel write performance with minimal changes to their codes.

https://github.com/DataLib-ECP/vol-log-based/blob/master/README.md

Kai Yuan Hou

(khl7265 at ece dot northwestern dot edu)

DAOS4004daos

Designed to utilize the DAOS object storage system by use of the DAOS API

 https://doi.org/10.1109/TPDS.2021.3097884

https://github.com/HDFGroup/vol-daos

HDF5 DAOS VOL Connector Design

HDF5 DAOS VOL Connector User's Guide

help at hdfgroup dot org
native0native  help at hdfgroup dot org
pass-through517pass_through_extProvides a simple example of a pass-through VOL connectorhttps://github.com/hpc-io/vol-external-passthrough

Suren Byna (sbyna at lbl dot gov)

dset-split518dset-split

Creates separate sub files for each dataset created and mounts these sub-files as external links in the main file. It enables versioning of HDF5 files at a dataset boundary.

https://github.com/hpc-io/vol-dset-splitAnnmary Justine (annmary dot roy at hpe dot com)
PDC-VOL519PDC-VOL

It is a terminal VOL that reads and writes HDF5 objects to the PDC system <https://github.com/hpc-io/pdc>

https://github.com/hpc-io/vol-pdcHoujun Tang (htang4 at lbl dot gov)

* The Search Name provides a mechanism for searching for a VOL.

 

List of Prototype VOL Connectors

 

ConnectorConnector IdentifierSearch Name*Short DescriptionURL

Contacts

radosunassignedradosPrototype VOL connector to access data in RADOShttps://github.com/HDFGroup/vol-radoshelp at hdfgroup dot org
RESTunassignedRESTDesigned to utilize web-based storage systems by use of the HDF5 REST APIshttps://github.com/HDFGroup/vol-resthelp at hdfgroup dot org

* The Search Name provides a mechanism for searching for a VOL.

--- Last Modified: January 19, 2022 | 12:14 PM