The purpose of this page is to list and briefly describe the documentation available to those who will use the Single-Writer/Multiple-Reader (SWMR) feature found in the HDF5 library. The following are the sections on this page:
SWMR User’s Guide
The HDF5 Single-Writer/Multiple-Reader User’s Guide describes HDF5’s SWMR feature and describes how to use the feature. The feature allows concurrent access of HDF5 files by writer and reader processes.
HDF5 Library APIs
| H5F_START_SWMR_WRITE | Enables SWMR writing mode for a file |
| H5DO_APPEND | Appends data to a dataset along a specified dimension |
| H5P_GET_APPEND_FLUSH | Retrieves the values of the append property that is set up in the dataset access property list |
| H5P_SET_APPEND_FLUSH | Sets two actions to perform when the size of a dataset’s dimension being appended reaches a specified boundary |
| H5P_GET_OBJECT_FLUSH_CB | Retrieves the object flush property values from the file access property list |
| H5P_SET_OBJECT_FLUSH_CB | Sets a callback function to invoke when an object flush occurs in the file |
| H5O_DISABLE_MDC_FLUSHES | Prevents metadata entries for an HDF5 object from being flushed from the metadata cache to storage |
| H5O_ENABLE_MDC_FLUSHES | Returns the cache entries associated with an HDF5 object to the default metadata flush and eviction algorithm |
| H5O_ARE_MDC_FLUSHES_DISABLED | Determines if an HDF5 object (dataset, group, committed datatype) has had flushes of metadata entries disabled |
Tools
The tools listed below will be incorporated into the HDF5 Reference Manual when they are fully released.
| h5watch | Allows users to output new records appended to a dataset as it grows In other words, this tool can be used to watch data that is added to a dataset. The functionality is similar to the Unix user command |
| h5format_convert | Allows users to convert the indexing type of a chunked dataset made with a 1.10.x version of the HDF5 library to the 1.8.x version 1 B-tree indexing type |
| h5clear | Allows users to clear the file consistency flags field in the superblock of a file |
SWMR Design Documents
| Metadata Cache Issues under SWMR and their Solutions | The HDF5 library contains a cache that provides more efficient access to HDF5 file metadata. Although helpful in most instances, this cache can cause problems when the single-writer/multiple-reader (SWMR) data access pattern is used to access an HDF5 file. Additionally, there are even deeper I/O issues that can affect HDF5 file metadata operations. This document describes the cache-oriented problems that the HDF5 library must overcome in order to implement the SWMR feature as well as our solutions. |
| Improvements for SWMR File Access and Dataset Append | This document describes changes to the HDF5 Library that improve the SWMR (Single-Writer/Multiple-Reader) file access model and provide better support for dataset append operations. |
| HDF5 Single-Writer/Multiple-Reader Feature Design and Semantics | This document describes the design and semantics of HDF5’s single-writer/multiple-reader (SWMR) feature. This feature allows multiple reader processes to inspect a file that is concurrently being written to by a single writer process without requiring any inter-process communication. |
| File Locking under SWMR – Semantics, Programming Model, and Implementation | This document describes the design and semantics of how file locking is used to help prevent inappropriate concurrent file access. This locking is only intended to help the user conform to SWMR semantics and is not a fundamental part of SWMR synchronization. |
--- Last Modified: January 02, 2019 | 10:58 AM