JPEG 2000 Interactive Protocol (JPIP) flexibility with respect to random access, code stream reordering and incremental decoding is highly exploitable in a networked environment allowing access to remote large files using limited bandwidth connections or high contention networks.
A brief overview of the JPIP event sequence is presented in this section, more information can be found at JPEG 2000 Interactive Protocol (Part 9 – JPIP) and the specification can (and should) be purchased from ISO.
An earlier version of JPEG 2000 Part 9 is available here http://www.jpeg.org/public/fcd15444-9v2.pdf, noting the ISO copyright, diagrams are not replicated in this documentation.
The JPIP protocol has been abstracted in this format driver, requests are made at the 1:1 resolution level.
The JPIPKAK driver uses an approach that was first demonstrated here, J2KViewer, by Juan Pablo Garcia Ortiz of separating the communication layer (socket / http) from the Kakadu kdu_cache object. Separating the communication layer from the data object is desirable since it allows the use of optimized http client libraries such as libcurl, Apache HttpClient (note that jportiz used a plain Java socket) and allows SSL communication between the client and server.
Kakadu's implementation of client communication with a JPIP server uses a socket, and this socket connection holds the state for this client session. A client session with Kakadu can be recreated using the JPIP cache operations between client and server, but no use of traditional HTTP cookies is supported since JPIP is neutral to the transport layer.
The JPIPKAK driver is written using a HTTP client library with the Kakadu cache object and supports optimized communication with a JPIP server (which may or may not support HTTP sessions) and the high performance of the kakadu kdu_region_decompressor.
The implementation supports the GDAL C++ and C API, and provides an initial SWIG wrapper for this driver with a Java ImageIO example (TODO - qGIS Example).
The driver uses a simple threading model to support requesting reads of the data and remote fetching. This threading model supports two separate client windows, with just one connection to the server. Requests to the server are multiplexed to utilize available bandwidth efficiently. The client identifies these windows by using “0” (low) or “1” (high) values to a “PRIORITY” metadata request option.
Note: SSL support
If the client is built with support for SSL, then driver determines whether to use SSL if the request is a jpips:// protocol as opposed to jpip:// . Note that the driver does not verify server certificates using the Curl certificate bundle and is currently set to accept all SSL server certificates.
Note: libCurl
JPIP sets client/server values using HTTP headers, modifications have been made to the GDAL HTTP portability library to support this.
Fetch the driver to which this dataset relates.
If the filename contained in the GDALOpenInfo object
has a case insensitive URI scheme of JPIP or JPIPS
the JPIPKAKDataset is created and initialized,
otherwise NULL is returned.
Initialization involves making an initial connection to the JPIP Server to establish a session and to retrieve the initial metadata about the image (ref. JPIP Sequence Diagram).
If the connection fails, the function returns false and the Open function returns NULL indicating that opening the dataset with this driver failed.
If the connection is successful, then subsequent requests to the JPIP server are made to retrieve all the available metadata about the image. Metadata items are set using the GDALMajorObject->SetMetadataItem in the "JPIP" domain.
If the metadata returned from the server includes GeoJP2 UUID box, or a GMLJP2 XML box then this metadata is parsed and sets the geographic metadata of this dataset.
C API to JPIPKAKDataset->GetMetadata
returns metadata for the "JPIP" domain, keys are "JPIP_NQUALITYLAYERS", "JPIP_NRESOLUTIONLEVELS", "JPIP_NCOMPS" and "JPIP_SPRECISION"
If the asynchronous raster IO is active and not required, the C API calls JPIPKAKDataset->EndAsyncRasterIO
The JPIPKAKAsyncRasterIO object is deleted
C API to JPIPKAKDataset->BeginAsyncRasterIO
The client has set the requested view window at 1:1 and have optionally set the discard level, quality layers and thread priority metadata items.
Creates a JPIPKAKAsyncRasterIO Object
Configures the kakadu machinery and starts a background thread (if not already running) to communicate to the server the current view window request. The background thread results in the kdu_cache object being updated until the JPIP server sends an "End Of Response" (EOR) message for the current view window request.
C API to LockBuffer
Not implemented in JPIPKAKAsyncRasterIO, a lock is acquired in JPIPKAKAsyncRasterIO->GetNextUpdatedRegion
C API to GetNextUpdatedRegion
The function decompresses the available data to generate an image (according to the dataset buffer type set in JPIPKAKDataset->BeginAsyncRasterIO)
The window width, height (at the requested discard level) decompressed is returned in the region pointer and can be rendered by the client.
The status of the rendering operation is one of GARIO_PENDING, GARIO_UPDATE, GARIO_ERROR, GARIO_COMPLETE from the GDALAsyncStatusType structure. GARIO_UPDATE, GARIO_PENDING require more reads of GetNextUpdatedRegion to get the full image data, this is the progressive rendering of JPIP. GARIO_COMPLETE indicates the window is complete.
GDALAsyncStatusType is a structure used byGetNextUpdatedRegion to indicate whether the function should be called again when either kakadu has more data in its cache to decompress, or the server has not sent an End Of Response (EOR) message to indicate the request window is complete.
The region passed into this function is passed by reference, and the caller can read this region when the result returns to find the region that has been decompressed. The image data is packed into the buffer, e.g. RGB if the region requested has 3 components.
C Api to UnlockBuffer
Not implemented in JPIPKAKAsyncRasterIO, a lock is acquired in JPIPKAKAsyncRasterIO->GetNextUpdatedRegion
Client renders image data
Currently only a Windows makefile is provided, however this should compile on Linux as well as there are no Windows dependencies.
See Also:
Driver originally developed by ITT VIS and donated to GDAL to enable SSL enabled JPIP client streaming of remote JPEG 2000 datasets.
