This page describes the new HDF5 Page Buffering option and provides links to available reference manual entries. The page includes the following sections:
Overview of the Page Buffering Feature
Page buffering adds a layer immediately above the Virtual File Driver I/O layer within the library, which acts as a page cache for metadata and raw data accesses that are smaller than the page size for file space allocation. For most applications the HDF5 library generates very small amounts of metadata and raw data accesses in HDF5 datasets, even though the total amount of metadata in an HDF5 file varies between applications. These small and random I/O accesses on parallel file systems result in poor performance for applications.
The page buffering feature, in conjunction with paged aggregation, can be used to reduce the number of small accesses in HDF5. Together these features give an application control over minimizing the granularity and alignment of HDF5 I/O requests. See File Space Management for information on paged aggregation.
See the RFC on this feature for complete details.
HDF5 Library APIs
Following are the new APIs introduced for the Page Buffering feature:
|H5P_GET_FILE_SPACE_STRATEGY||Retrieves the File Space Strategy for a file creation property list. See File Space Management for more details|
|H5P_GET_FILE_SPACE_PAGE_SIZE||Retrieves the file space page size|
|H5P_SET_FILE_SPACE_PAGE_SIZE||Sets the file space page size when file space allocation is |
|H5P_GET_PAGE_BUFFER_SIZE||Retrieves the maximum size for the page buffer and the minimum percentage for metadata and raw data pages|
|H5P_SET_PAGE_BUFFER_SIZE||Sets the maximum size for the page buffer and the minimum percentage for metadata and raw data pages|
|H5F_GET_PAGE_BUFFERING_STATS||Gathers page buffer statistics regarding page access when it is enabled|
|H5F_RESET_PAGE_BUFFERING_STATS||Resets the page buffer statistics|
|The following options were added to |
--- Last Modified: April 06, 2018 | 02:36 PM