Advanced Topics in HDF5

The following topics are broadly applicable in HDF5 applications.

• Dynamic Plugins in HDF5 - Describes the use of dynamic plugins, including dynamically loaded filters, in HDF5. These plugins need not be registered with the HDF5 library.
• Using the Direct Chunk Write Function - Describes another way that chunks can be written to datasets
• Copying Committed Datatypes with H5Ocopy - Describes how to copy to another file a dataset that uses a committed datatype or an object with an attribute that uses a committed datatype so that the committed datatype in the destination file can be used by multiple objects
• HDF5 Metadata - Provides a comprehensive overview of the types of metadata used in HDF5
• HDF5 Data Flow Pipeline for H5Dread - Describes data flow when reading raw data from an HDF5 dataset
• Virtual Dataset - VDS - Describes how to present data stored in several HDF5 datasets and files as a single dataset and access the data using HDF5 APIs (HDF5-1.10)
• Virtual Object Layer - An abstraction layer within the HDF5 library that enables different methods for accessing data and objects that conform to the HDF5 data model

The following topics are of interest in certain limited situations such as when space utilization or performance must be carefully tuned.

• Collective Metadata I/O  - Describes optimizing HDF5 by collecting metadata I/O (HDF5-1.10)
• Metadata Caching in HDF5 - Provides detailed information for managing and tuning HDF5 metadata caching
• HDF5 File Image Operations   - Describes how to work with HDF5 files in memory. Disk I/O is not required when file images are opened, created, read from, or written to
• Modified Region Writes   - Describes how to set write operations for in-memory files so that only modified regions are written to storage. Available when the Core (Memory) VFD is used.
• Freeing Memory Allocated by the HDF5 Library   - Describes how inconsistent memory management can cause heap corruption or resource leaks and possible solutions
• File Space Management  -  Describes mechanisms to manage space in an HDF5 file (HDF5-1.10)
• Fine-tuning the Metadata Cache  - Describes how to provide better control of the metadata cache (HDF5-1.10)
• Enabling a Strict Consistency Semantics Model in Parallel HDF5   - Describes how to improve file access consistency semantics with the Parallel HDF5 library
• Page Buffering - Describes how to reduce the number of small accesses in HDF5 by buffering metadata and raw data accesses (HDF5-1.10)
• Partial Edge Chunks  - Describes how to control the filtering of partial edge chunks to improve performance when extending datasets (HDF5-1.10)
• Single Writer/Multiple-Reader - Enables writing to a file while multiple readers are accessing it (HDF5-1.10)
• Thread-safe HDF5 - Information on thread-safety and concurrent access in HDF5