Hierarchical data format (HDF5) for Modflow, Modpath and ZoneBudget

Abstract

More and more spatially detailed time dependent groundwater models are used and hence input files for models like Modflow (USGS) are becoming larger and larger. These input files are in ASCII format and contain datasets with values for each cell, changing often for each stress period. To diminish the problem of huge ASCII files, the datasets of the input files can be stored in an HDF5 file (Hierarchical data format). HDF5 is a data model, library and file format designed by HDFgroup for storing and managing data, flexible and efficient input and output and high volume and complex data. The file has a binary format and can be compressed with different kinds of compression methods. An HDF5 file consists among others of groups and datasets, referencing a dataset in the HDF5 file is similar to referencing a file in a folder of a file system. The contents of an HDF5 file can be viewed with HDFview, a Java-based viewer. HDF5 files can be constructed manually with the help of HDFview, or with the help of programming languages like C, C++, Matlab, Fortran or Python. The goal of this contribution is to show how HDF can improve data management of Modflow. A similar HDF5 link to Modflow has been implemented in the interface Groundwater Modeling System. Modflow has been extended to be able to read datasets from the ASCII input files which contain a reference to an HDF5 file for every real 1 or 2-dimensional dataset, every 2-dimensional integer dataset and the stress lists described in the Modflow user manual as U1DREL, U2DREL, U2DINT and ULSTRD. Two other programs from USGS, Modpath and Zonebudget, are also using partly the same input files as Modflow, and have been extended to be able to read the requested datasets from the HDF5 file. The total Modflow input file size, i.e. converted ASCII files and HDF5 file, will be decimated compared to the original size. Partly this is due to 'zlib' compression, 'zlib' is a free lossless data compression library. Due to faster reading of the HDF5 data compared to the original ASCII data model the Modflow, Modpath and ZoneBudget run time decreases.

Publication:

Pub Date:

April 2012

Bibcode:

2012EGUGA..14.8028C

Hdf Zlib File Size

I am trying to use a newer version of zlib (which I compiled myself) with hdf5-1.8.20, using pgfortran. = -1 ENOENT (No such file or directory) open('/usr/local. OK, I recompiled using make V=1 and re-directed to a file. Here’s my current compile script: cat my.configure #!/bin/csh -f. Make distclean & /dev/null. Setenv FC pgfortran. Sep 03, 2014 -Russ address@hidden wrote: Hi BenFrom the file you provided, I.believe. that you are using an hdf5 that was built without zlib support. I have a small class that I use to manage some data that I'm working with. I use it to translate a 'hdf path' in to an actual on-disk file since storing all the data in one file just ends up being a mess that keeps growing as I debug.