H5P_SET_NBIT sets the N-Bit filter, H5Z_FILTER_NBIT
, in the dataset creation property list plist_id
.
The HDF5 user can create an N-Bit datatype with the following code:
hid_t nbit_datatype = H5Tcopy(H5T_STD_I32LE);
H5Tset_precision(nbit_datatype, 16);
H5Tset_offset(nbit_datatype, 4);
In memory, one value of the N-Bit datatype in the above example will be stored on a little-endian machine as follows:
byte 3 | byte 2 | byte 1 | byte 0 |
???????? | ????SPPP | PPPPPPPP | PPPP???? |
Note: S - sign bit, P - significant bit, ? - padding bit; For signed integer, the sign bit is included in the precision.
When data of the above datatype is stored on disk using the N-bit filter, all padding bits are chopped off and only significant bits are stored. The values on disk will be something like:
1st value | 2nd value | |
SPPPPPPPPPPPPPPP | SPPPPPPPPPPPPPPP | ... |
The N-Bit filter is used effectively for compressing data of an N-Bit datatype as well as a compound and an array datatype with N-Bit fields. However, the datatype classes of the N-Bit datatype or the N-Bit field of the compound datatype or the array datatype are limited to integer or floating-point.
The N-Bit filter supports complex situations where a compound datatype contains member(s) of a compound datatype or an array datatype that has a compound datatype as the base type. However, it does not support the situation where an array datatype has a variable-length or variable-length string as its base datatype. The filter does support the situation where a variable-length or variable-length string is a member of a compound datatype.
The N-Bit filter allows all other HDF5 datatypes (such as time, string, bitfield, opaque, reference, enum, and variable length) to pass through as a no-op.
Like other I/O filters supported by the HDF5 library, application using the N-Bit filter must store data with chunked storage.
By nature, the N-Bit filter should not be used together with other I/O filters.