This page describes the new HDF5 Page Buffering option and provides links to available reference manual entries. The page includes the following sections:
Overview of the Page Buffering Feature
Summary
Page buffering adds a layer immediately above the Virtual File Driver I/O layer within the library, which acts as a page cache for metadata and raw data accesses that are smaller than the page size for file space allocation. For most applications the HDF5 library generates very small amounts of metadata and raw data accesses in HDF5 datasets, even though the total amount of metadata in an HDF5 file varies between applications. These small and random I/O accesses on parallel file systems result in poor performance for applications.
The page buffering feature, in conjunction with paged aggregation, can be used to reduce the number of small accesses in HDF5. Together these features give an application control over minimizing the granularity and alignment of HDF5 I/O requests. See File Space Management for information on paged aggregation.
Background Information
See the RFC on this feature for complete details.
HDF5 Library APIs
Following are the new APIs introduced for the Page Buffering feature:
| H5P_GET_FILE_SPACE_STRATEGY | Retrieves the File Space Strategy for a file creation property list. See File Space Management for more details |
| H5P_SET_FILE_SPACE_STRATEGY | H5F_FSPACE_STRATEGY_PAGE sets the file space handling strategy to use Page Buffering. See File Space Management for more details |
| H5P_GET_FILE_SPACE_PAGE_SIZE | Retrieves the file space page size |
| H5P_SET_FILE_SPACE_PAGE_SIZE | Sets the file space page size when file space allocation is H5F_FSPACE_STRATEGY_PAGE |
| H5P_GET_PAGE_BUFFER_SIZE | Retrieves the maximum size for the page buffer and the minimum percentage for metadata and raw data pages |
| H5P_SET_PAGE_BUFFER_SIZE | Sets the maximum size for the page buffer and the minimum percentage for metadata and raw data pages |
| H5F_GET_PAGE_BUFFERING_STATS | Gathers page buffer statistics regarding page access when it is enabled |
| H5F_RESET_PAGE_BUFFERING_STATS | Resets the page buffer statistics |
Tools
h5repack | The following options were added to h5repack:
|
--- Last Modified: April 06, 2018 | 02:36 PM