What it is
The Java HDF Interface (JHI) is a Java package (hdf.hdflib) that ``wraps'' the HDF-4 library*. The JHI may be used by any Java application that needs to access HDF-4 files. This product cannot be used in most network browsers because it accesses the local disk using native code.
...
The Java HDF Interface consists of Java classes and dynamically linked native libraries. The Java classes declare static native methods, and the library contains C functions which implement the native methods. The C functions call the standard HDF library, which is linked as part of the same library on most platforms.
Intended purpose
The Java HDF Interface is intended to be the standard interface for accessing HDF-4 files from Java programs. The JHI is a foundation upon which application classes can be built. All classes that use this interface package should interoperate easily on any platform that has HDF-4 installed.
It is unlikely that Java programs will want to directly call the HDF library. More likely, data will be represented by Java classes that meet the needs of the application. These classes can implement methods to store and retrieve data from HDF files using the Java HDF library.
Important Changes
| Very Important Change: Version 3.0 (and above) of the JHI packages all HDF library calls as "hdf.hdflib", note that the "ncsa" has been removed. Source code which used earlier versions of the JHI should be changed to reflect this new implementation. |
...
- Support for Macintosh and PC path names, e.g., in Hopen().
- A separate distribution of the JHI. This contains everything from the full source distribution, omitting all the packages not used by the JHI.
- The HDFArray class was extended to support more types of Java arrays, including arrays of any sub-class of Number (Integer, Float, etc.) and arrays of String.
...
How to use it
For example, the HDF library had the function Hopen to open an HDF file. The Java interface is the class hdf.hdflib.HDFLibrary, which has a method:
...
The HDFLibrary class automatically loads the HDF-4 library with the native method implementations and the HDF-4 library.
...
Technical notes
The following is a list of a few important technical notes of the JHI. For more details, please read JHI Design Notes.
...
- Exceptions. The JHI declares all the methods of HDF to throw Java exceptions of type `HDFLibraryException'. Errors returned by the native HDF library will be detected and the appropriate Exception raised, just as from regular Java methods. There is no need for the Java program to analyze HDF library error codes or return values. However, in this release many HDF library errors are not detected as exceptions, so exceptions will not be raised except when opening a file. Error handling will be completed in a future release.
...
| Anchor | ||||
|---|---|---|---|---|
|
Important Note
This documentation has not been updated to reflect changes since 1997.
Most of the information is correct, but please see the release notes for important changes
07/06/2016
Abstract
Applications access Hierarchical Data Format (HDF) files through the HDF library code. In the absence of a Java(TM) implementation of the HDF library (an extremely remote contingency), Java applications that need to directly access data in HDF files require a Java interface to the native code HDF library. This document describes the design of the Java interface for the HDF library.
1. Introduction
The current release of the Hierachical Data Format API (HDF 4) is not a simple or consistent object model. It consists of a dozen or so ``Interfaces'', with different models of data and data access. This reflects the history of development, as new features have been introduced while older features have been maintained for compatibility. The size and complexity of HDF also reflects the diversity of its uses and users. Different communities and applications use particular parts of the HDF API. In this sense, HDF represents the ``union'' of several data models.
...
Because the HDF API is not object oriented, creating Java classes to access the HDF library is not trivial. It is first necessary to create an object model for HDF, which might be done in many ways. Further, different communities or applications will need appropriate object models, e.g., for imagery or multi-dimensional grids. It is not always easy to discern the border between the abstractions HDF should provide and what should be left to the application program. This document describes a standard set of Java classes to access HDF files, closely modelling the HDF API. This HDF Java Interface forms a solid foundation for constructing and--more importantly--sharing problem specific data models, as part of the HDF distribution or from other parties.
1.1 Previous work
In previous work, we have implemented a Java based browser for HDF files, called the Java HDF Viewer (JHV). Java applets generally cannot access files or use native code libraries (such as HDF), so the JHV is implemented as a Java application, that is, as a free standing program, not a network loadable applet. The JHV application links to the standard HDF library, through the Java ``native code'' methods to read HDF files on local disk. The lastest release of the JHV uses the HDF Java Interface described here.
...
The initial implementation of the JHV implemented a different Java-to-C wrapper layer. In the initial implementation, only parts of the HDF API were included, specifically, routines needed by the JHV to inquire about metadata and to read data. No write functions were implemented, and many miscellaneous access functions were omitted because they were not needed.
1.2 The Java Development Kit (JDK)
The first implementation of the JHV was based on the JDK 1.0.2. This early release of the Java environment did not support Java-to-C interfaces very well. The interfacing mechanism was incomplete, poorly documented, and (most critically) not standard across platforms. Fortunately, many of the shortcomings of JDK 1.0.2 were addressed by major improvements in JDK 1.1.
...
Of particular importance to this effort was the Array class, which allows the discovery of the shape, size, and type of arbitrary Java array objects, and also supports the creation and manipulation of Java array objects. This package makes it possible to store and retrieve multi-dimensional Java array objects using HDF.
2. The Object Model
As noted, there are potentially many ways to model access to the data object of HDF. The model described here chooses to model the HDF library itself. Thus, the central object in this model is the ``HDF'' object, which has 300 some native methods, corresponding to all the operations provided by the HDF native code library. The purpose of this HDF library object is to provide the foundation for any other object model which wishes to use data in HDF files. It is likely that most Java applications will not directly use the HDF object; rather will use more abstract object models which themselves use HDF to manage and access storage. This is analogous to the role of HDF-EOS: programs create and manipulate HDF-EOS data objects, not the HDF representations underlying them.
2.1 The principle objects
[...need O diagram here]
The objects of the model are:
...
Our implementation of these classes makes it possible for Java classes to perform essentially any operation that is supported by the HDF library. The Java code to access HDF is very similar to Fortran or C -- a sequence of calls to the routines of the HDF interface. Of course, Java programs will generally want to encapsulate these code sequences in more abstract classes, which represent the data objects of interest.
2.2 Use of the HDF Java classes
The HDF Java Interface classes are intended to be used in two ways:
...
There have been many ideas for both general and application specific object models for data and images. The HDF Java classes described here are the necessary foundation that allows these kinds of models to be implemented quickly and portably using Java, with the ability to create and use data stored in HDF.
3. Implementation
An initial implementation of the HDF Java classes has been completed. The implementation required solution of a number of important problems that are of general interest to any effort to interface Java and C. These include:
...
These issues are discussed in this section.
3.1 Data Transfer and Translation Issues
Many HDF functions return several data items in addition to a return value. That is, many of the parameters to the routines of the HDF library are ``OUT'' or ``IN/OUT'' parameters. In C this is expressed by call-by-reference semantics, that is, using pointers. Java passes all arguments by value, so special operations are required to return a value through a parameter. The C code must invoke operations on arrays, or invoke methods of the other types of objects.
...
Clearly the code to manipulate C structures is awkward and inefficient, especially if reasonable error checking is added. It should be noted that this is scarcely an unprecedented problem: the Fortran interface to HDF has had to implement similar translations, as Fortran cannot directly implement many C data structures. Aside from changing the HDF API, there is no alternative to this kind of translation when attempting to provide multi-language support.
3.2 Performance
All the data conversions discussed above involve data copying. As far as I can determine, these copies are unavoidable. In fact, the majority of the data read or written through the Java HDF API is copied by the Java-to-C code for one reason or another. The actual amount of copying that occurs may vary from platform to platform, as the JNI operations may be implemented through additional (albeit hidden) data copies on some architectures (e.g., to assure correct alignment or byte order between the Java VM and the native machine.)
...
While the garbage collector is running, less useful computation is accomplished. This overhead appears to be unavoidable. The Java memory management model is designed to be simple, secure, and portable. This works acceptably for reasonable numbers of reasonable sized objects, but becomes a performance problem for data intensive programs handling large amounts of memory. So again, this work has shown how important positive assets of Java have inherent performance consequences.
3.4 Exceptions
The Java language provides a clean, state of the art, mechanism for raising and handling exceptions. C has no such mechanism, but the JNI provides access to the Java model. This support allows the Java-to-C code to detect errors reported by the native code library, and then to create and raise appropriate exceptions. This is actually a fairly elegant solution, allowing the Java code to be written in a natural style, despite the fact that native code is involved.
...
- A more elaborate exception model, with more subclasses of Exceptions reflecting different categories of errors. This is especially attractive if some classes of errors can be automatically handled by some programs, perhaps adjusting the parameters to fit the actual range, breaking up large requests into a series of smaller ones.
- Perhaps add code to handle some exceptions in Java interface code itself.
- More elaborate reporting, especially when HDF returns a stack of errors. In general, the error conditions from HDF could be analyzed and filtered, with the HDF Java Interface either masking some failures (e.g., through retries or reasonable defaults) or raising very specific exceptions. This would reduce the need for the application itself to process the exceptions returned.
4. Summary and Conclusions
The implementation described in this paper shows that the JDK 1.1 contains most of the features needed to interface even to a complex, non-object oriented native library such as HDF. However, the HDF Java Interface is far from trivial, and it remains to be seen if it will be useful for many applications.
...
The performance of Java may limit its use for some aspects of scientific computing. For instance, it is unlikely that Java would be useful for the inner loop of a large numeric computation. However, Java will be very useful for portable interfaces, remote data access, and for visualization. It is likely that there will be many hybrid environments that use Java as a framework for interactive computing, liking to specialized native code applications and libraries for efficient computation. Here again, the ability to use HDF will be valuable as a means to exchange data among programs written in different languages, as well as running on different networks.
Acknowledgements
This work was supported by Project Horizon, a cooperative agreement between the University of Illinois and NASA. (OLD URL: http://horizon.ncsa.uiuc.edu/horizon/).
Other funding was provided by the Digital Library Initiative at the University of Illinois. ( http://dli.grainger.uiuc.edu) The UIUC DLI is a recipient of a grant in the NSF/DARPA/NASA/Digital Libraries Initiative.
Figures
intn SDfileinfo(int32 sd_id, int32 *ndatasets, int32 *nglobal_attr)where the parameters are: sd_id IN: The SD interface identifier returned from SDstart ndatasets OUT: Number of data sets in the file nglobal_attr OUT: Number of global attributes in the file |
...