Please, help us to better serve our user community by answering the following short survey: https://www.hdfgroup.org/website-survey/
HDF5  1.14.4.3
API Reference
 
Loading...
Searching...
No Matches
Metadata Cache

Detailed Description

Functions

herr_t H5Fget_mdc_config (hid_t file_id, H5AC_cache_config_t *config_ptr)
 Obtains current metadata cache configuration for target file.
 
herr_t H5Fset_mdc_config (hid_t file_id, const H5AC_cache_config_t *config_ptr)
 Attempts to configure metadata cache of target file.
 
herr_t H5Fget_mdc_hit_rate (hid_t file_id, double *hit_rate_ptr)
 Obtains target file's metadata cache hit rate.
 
herr_t H5Fget_mdc_size (hid_t file_id, size_t *max_size_ptr, size_t *min_clean_size_ptr, size_t *cur_size_ptr, int *cur_num_entries_ptr)
 Obtains current metadata cache size data for specified file.
 
herr_t H5Freset_mdc_hit_rate_stats (hid_t file_id)
 Resets hit rate statistics counters for the target file.
 
herr_t H5Fstart_mdc_logging (hid_t file_id)
 Starts logging metadata cache events if logging was previously enabled.
 
herr_t H5Fstop_mdc_logging (hid_t file_id)
 Stops logging metadata cache events if logging was previously enabled and is currently ongoing.
 
herr_t H5Fget_mdc_logging_status (hid_t file_id, hbool_t *is_enabled, hbool_t *is_currently_logging)
 Gets the current metadata cache logging status.
 
herr_t H5Fget_mdc_image_info (hid_t file_id, haddr_t *image_addr, hsize_t *image_size)
 Obtains information about a cache image if it exists.
 

Function Documentation

◆ H5Fget_mdc_config()

herr_t H5Fget_mdc_config ( hid_t  file_id,
H5AC_cache_config_t config_ptr 
)

Obtains current metadata cache configuration for target file.

Parameters
[in]file_idFile identifier
[in,out]config_ptrPointer to the H5AC_cache_config_t instance in which the current metadata cache configuration is to be reported. The fields of this structure are discussed here.
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.
Note
The in direction applies only to the H5AC_cache_config_t::version field. All other fields are out parameters.

H5Fget_mdc_config() loads the current metadata cache configuration into the instance of H5AC_cache_config_t pointed to by the config_ptr parameter.
The fields of the H5AC_cache_config_t structure are shown below:

typedef struct H5AC_cache_config_t {
/* general configuration fields: */
int version;
size_t initial_size;
size_t max_size;
size_t min_size;
long int epoch_length;
/* size increase control fields: */
double increment;
size_t max_increment;
/* size decrease control fields: */
double decrement;
size_t max_decrement;
double empty_reserve;
/* parallel configuration fields: */
#define H5AC__MAX_TRACE_FILE_NAME_LEN
Definition H5ACpublic.h:432
H5C_cache_incr_mode
Definition H5Cpublic.h:26
H5C_cache_decr_mode
Definition H5Cpublic.h:42
H5C_cache_flash_incr_mode
Definition H5Cpublic.h:34
bool hbool_t
Definition H5public.h:249
Definition H5ACpublic.h:462
enum H5C_cache_flash_incr_mode flash_incr_mode
Definition H5ACpublic.h:615
double lower_hr_threshold
Definition H5ACpublic.h:585
hbool_t apply_max_increment
Definition H5ACpublic.h:607
hbool_t evictions_enabled
Definition H5ACpublic.h:526
double decrement
Definition H5ACpublic.h:657
enum H5C_cache_decr_mode decr_mode
Definition H5ACpublic.h:638
size_t initial_size
Definition H5ACpublic.h:550
hbool_t open_trace_file
Definition H5ACpublic.h:478
hbool_t apply_empty_reserve
Definition H5ACpublic.h:683
hbool_t set_initial_size
Definition H5ACpublic.h:546
size_t max_decrement
Definition H5ACpublic.h:673
double flash_multiple
Definition H5ACpublic.h:620
int epochs_before_eviction
Definition H5ACpublic.h:677
int metadata_write_strategy
Definition H5ACpublic.h:707
double upper_hr_threshold
Definition H5ACpublic.h:643
size_t dirty_bytes_threshold
Definition H5ACpublic.h:696
char trace_file_name[1024+1]
Definition H5ACpublic.h:511
double flash_threshold
Definition H5ACpublic.h:629
double empty_reserve
Definition H5ACpublic.h:687
hbool_t rpt_fcn_enabled
Definition H5ACpublic.h:470
int version
Definition H5ACpublic.h:465
double min_clean_fraction
Definition H5ACpublic.h:556
hbool_t apply_max_decrement
Definition H5ACpublic.h:669
double increment
Definition H5ACpublic.h:595
long int epoch_length
Definition H5ACpublic.h:573
hbool_t close_trace_file
Definition H5ACpublic.h:499
size_t max_increment
Definition H5ACpublic.h:611
enum H5C_cache_incr_mode incr_mode
Definition H5ACpublic.h:580
size_t max_size
Definition H5ACpublic.h:565
size_t min_size
Definition H5ACpublic.h:569

(Click on a enumerator, field, or type for more information.)

Since
1.8.0

◆ H5Fget_mdc_hit_rate()

herr_t H5Fget_mdc_hit_rate ( hid_t  file_id,
double *  hit_rate_ptr 
)

Obtains target file's metadata cache hit rate.

Parameters
[in]file_idFile identifier
[out]hit_rate_ptrPointer to the double in which the hit rate is returned. Note that hit_rate_ptr is undefined if the API call fails
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fget_mdc_hit_rate() queries the metadata cache of the target file to obtain its hit rate (cache hits / (cache hits + cache misses)) since the last time hit rate statistics were reset. If the cache has not been accessed since the last time the hit rate stats were reset, the hit rate is defined to be 0.0.

The hit rate stats can be reset either manually (via H5Freset_mdc_hit_rate_stats()), or automatically. If the cache's adaptive resize code is enabled, the hit rate stats will be reset once per epoch. If they are reset manually as well, the cache may behave oddly.

See the overview of the metadata cache in the special topics section of the user manual for details on the metadata cache and its adaptive resize algorithms.

Since
1.8.0

◆ H5Fget_mdc_image_info()

herr_t H5Fget_mdc_image_info ( hid_t  file_id,
haddr_t image_addr,
hsize_t image_size 
)

Obtains information about a cache image if it exists.

Parameters
[in]file_idFile identifier
[out]image_addrOffset of the cache image if it exists, or HADDR_UNDEF if it does not
[out]image_sizeLength of the cache image if it exists, or 0 if it does not
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fget_mdc_image_info() returns information about a cache image if it exists.

When an HDF5 file is opened in Read/Write mode, any metadata cache image will be read and deleted from the file on the first metadata cache access (or, if persistent free space managers are enabled, on the first file space allocation / deallocation, or read of free space manager status, whichever comes first).

Thus, if the file is opened Read/Write, H5Fget_mdc_image_info() should be called immediately after file open and before any other operation. If H5Fget_mdc_image_info() is called after the cache image is loaded, it will correctly report that no cache image exists, as the image will have already been read and deleted from the file. In the Read Only case, the function may be called at any time, as any cache image will not be deleted from the file.

Since
1.10.1

◆ H5Fget_mdc_logging_status()

herr_t H5Fget_mdc_logging_status ( hid_t  file_id,
hbool_t is_enabled,
hbool_t is_currently_logging 
)

Gets the current metadata cache logging status.

Parameters
[in]file_idFile identifier
[out]is_enabledWhether logging is enabled
[out]is_currently_loggingWhether events are currently being logged
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

The metadata cache is a central part of the HDF5 library through which all file metadata reads and writes take place. File metadata is normally invisible to the user and is used by the library for purposes such as locating and indexing data. File metadata should not be confused with user metadata, which consists of attributes created by users and attached to HDF5 objects such as datasets via H5A API calls.

Due to the complexity of the cache, a trace/logging feature has been created that can be used by HDF5 developers for debugging and performance analysis. The functions that control this functionality will normally be of use to a very limited number of developers outside of The HDF Group. The functions have been documented to help users create logs that can be sent with bug reports.

Control of the log functionality is straightforward. Logging is enabled via the H5Pset_mdc_log_options() function, which will modify the file access property list used to open or create a file. This function has a flag that determines whether logging begins at file open or starts in a paused state. Log messages can then be controlled via the H5Fstart_mdc_logging() and H5Fstop_mdc_logging() functions. H5Pget_mdc_log_options() can be used to examine a file access property list, and H5Fget_mdc_logging_status() will return the current state of the logging flags.

The log format is described in the Metadata Cache Logging document.

Note
Unlike H5Fstart_mdc_logging() and H5Fstop_mdc_logging(), this function can be called on any open file identifier.
Since
1.10.0

◆ H5Fget_mdc_size()

herr_t H5Fget_mdc_size ( hid_t  file_id,
size_t *  max_size_ptr,
size_t *  min_clean_size_ptr,
size_t *  cur_size_ptr,
int *  cur_num_entries_ptr 
)

Obtains current metadata cache size data for specified file.

Parameters
[in]file_idFile identifier
[out]max_size_ptrPointer to the location in which the current cache maximum size is to be returned, or NULL if this datum is not desired
[out]min_clean_size_ptrPointer to the location in which the current cache minimum clean size is to be returned, or NULL if that datum is not desired
[out]cur_size_ptrPointer to the location in which the current cache size is to be returned, or NULL if that datum is not desired
[out]cur_num_entries_ptrPointer to the location in which the current number of entries in the cache is to be returned, or NULL if that datum is not desired
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fget_mdc_size() queries the metadata cache of the target file for the desired size information, and returns this information in the locations indicated by the pointer parameters. If any pointer parameter is NULL, the associated data is not returned.

If the API call fails, the values returned via the pointer parameters are undefined.

If adaptive cache resizing is enabled, the cache maximum size and minimum clean size may change at the end of each epoch. Current size and current number of entries can change on each cache access.

Current size can exceed maximum size under certain conditions. See the overview of the metadata cache in the special topics section of the user manual for a discussion of this.

Since
1.8.0

◆ H5Freset_mdc_hit_rate_stats()

herr_t H5Freset_mdc_hit_rate_stats ( hid_t  file_id)

Resets hit rate statistics counters for the target file.

Parameters
[in]file_idFile identifier
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

H5Freset_mdc_hit_rate_stats() resets the hit rate statistics counters in the metadata cache associated with the specified file.

If the adaptive cache resizing code is enabled, the hit rate statistics are reset at the beginning of each epoch. This API call allows you to do the same thing from your program.

The adaptive cache resizing code may behave oddly if you use this call when adaptive cache resizing is enabled. However, the call should be useful if you choose to control metadata cache size from your program.

See Metadata Caching in HDF5 for details about the metadata cache and the adaptive cache resizing algorithms. If you have not read, understood, and thought about the material covered in that documentation, you should not be using this API call.

Since
1.8.0

◆ H5Fset_mdc_config()

herr_t H5Fset_mdc_config ( hid_t  file_id,
const H5AC_cache_config_t config_ptr 
)

Attempts to configure metadata cache of target file.

Parameters
[in]file_idFile identifier
[in,out]config_ptrPointer to the H5AC_cache_config_t instance containing the desired configuration. The fields of this structure are discussed here.
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fset_mdc_config() attempts to configure the file's metadata cache according configuration supplied in config_ptr.

(Click on a enumerator, field, or type for more information.)

Since
1.8.0

◆ H5Fstart_mdc_logging()

herr_t H5Fstart_mdc_logging ( hid_t  file_id)

Starts logging metadata cache events if logging was previously enabled.

Parameters
[in]file_idFile identifier
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

The metadata cache is a central part of the HDF5 library through which all file metadata reads and writes take place. File metadata is normally invisible to the user and is used by the library for purposes such as locating and indexing data. File metadata should not be confused with user metadata, which consists of attributes created by users and attached to HDF5 objects such as datasets via H5A API calls.

Due to the complexity of the cache, a trace/logging feature has been created that can be used by HDF5 developers for debugging and performance analysis. The functions that control this functionality will normally be of use to a very limited number of developers outside of The HDF Group. The functions have been documented to help users create logs that can be sent with bug reports.

Control of the log functionality is straightforward. Logging is enabled via the H5Pset_mdc_log_options() function, which will modify the file access property list used to open or create a file. This function has a flag that determines whether logging begins at file open or starts in a paused state. Log messages can then be controlled via the H5Fstart_mdc_logging() and H5Fstop_mdc_logging() functions. H5Pget_mdc_log_options() can be used to examine a file access property list, and H5Fget_mdc_logging_status() will return the current state of the logging flags.

The log format is described in the Metadata Cache Logging document.

Note
Logging can only be started or stopped if metadata cache logging was enabled via H5Pset_mdc_log_options().
When enabled and currently logging, the overhead of the logging feature will almost certainly be significant.
The log file is opened when the HDF5 file is opened or created and not when this function is called for the first time.
This function opens the log file and starts logging metadata cache operations for a particular file. Calling this function when logging has already been enabled will be considered an error.
Since
1.10.0

◆ H5Fstop_mdc_logging()

herr_t H5Fstop_mdc_logging ( hid_t  file_id)

Stops logging metadata cache events if logging was previously enabled and is currently ongoing.

Parameters
[in]file_idFile identifier
Returns
Returns a non-negative value if successful; otherwise, returns a negative value.

The metadata cache is a central part of the HDF5 library through which all file metadata reads and writes take place. File metadata is normally invisible to the user and is used by the library for purposes such as locating and indexing data. File metadata should not be confused with user metadata, which consists of attributes created by users and attached to HDF5 objects such as datasets via H5A API calls.

Due to the complexity of the cache, a trace/logging feature has been created that can be used by HDF5 developers for debugging and performance analysis. The functions that control this functionality will normally be of use to a very limited number of developers outside of The HDF Group. The functions have been documented to help users create logs that can be sent with bug reports.

Control of the log functionality is straightforward. Logging is enabled via the H5Pset_mdc_log_options() function, which will modify the file access property list used to open or create a file. This function has a flag that determines whether logging begins at file open or starts in a paused state. Log messages can then be controlled via the H5Fstart_mdc_logging() and H5Fstop_mdc_logging() functions. H5Pget_mdc_log_options() can be used to examine a file access property list, and H5Fget_mdc_logging_status() will return the current state of the logging flags.

The log format is described in the Metadata Cache Logging document.

Note
Logging can only be started or stopped if metadata cache logging was enabled via H5Pset_mdc_log_options().
This function only suspends the logging operations. The log file will remain open and will not be closed until the HDF5 file is closed.
Since
1.10.0