U.S. JGOFS Merged Data Products Description
For a long time, the duties of the U.S. JGOFS Data Management Office (DMO) centered primarily
around the receipt and quality control of field data. This data is currently available
online. To facilitate the
Synthesis and Modeling
phase of the U.S. JGOFS Program, the DMO staff began generating 'merged
data products'. The merging process is an attempt to 'put the ocean back together again' after all the samples have been analyzed by investigators participating in the U.S. JGOFS program.
What is a merged data product?
A merged product is a large data set; a combination of data records that have
related variables and a common collection method. Initially,
the DMO produced both CTD and niskin bottle merged data products for the
Arabian Sea and Southern Ocean process studies. Merged data products for
the Equatorial Pacific Process Study and North Atlantic Bloom
Experiment as well as Trace Metal bottle samples were released next.
The process starts with the merging of related data objects from a
given cruise into a single cruise-level object. Cruise-level objects are then
merged into basin-level objects. Although the production of
global-level merged data products is possible, there is no
plan to do so at the moment because of the disparate time periods
In building the merged data products, we are taking advantage of
features of the database management system
developed for U.S.
JGOFS by Glenn Flierl of Massachusetts Institute of Technology, James
Bishop of Lawrence Berkeley National Laboratory and David Glover of
Woods Hole Oceanographic Institution
in the early days of the JGOFS program. A list is compiled of all data
from the field study database that should be combined into a single merged product.
The DBMS "join" function, then refers to this list and creates the merged product.
Logically the merged data products are not so much a union of data
objects as they are a projection of the smaller data object into a
larger data space. All data that go into merged objects must have
been collected by the same sampling device with the same methodology.
In special cases two or more merged products will be combined. For example,
it will be desirable to combine the Niskin Bottle and Trace Metal merged products for
investigators who collected water from both of these sampling systems.
After all data from the same sampling device have been
identified at the cruise level, the merge commences.
If event, station and bottle numbers are the same, data are
placed in the same row. If variable name and methodology are the
same, data are placed in the same column. An audit file keeps track of
any mismatched attributes, such as units, and the DMO staff resolves the mismatch
and remakes the merged product.
of the merging process
Data access via JGOFS customized Live Access Server
The increased volume of these synthesized data products requires
a new access interface and visualization tool for these larger data entities.
The DMO is working with the
Live Access Server (LAS)
group at the the University of Washington and the National
Oceanic and Atmospheric Administration (NOAA) Pacific Marine
Environmental Laboratory in Seattle to develop a U.S. JGOFS customized LAS (J-LAS).
The LAS was not originally designed for work with non-gridded data
sets. In a fast-moving and exciting collaboration, the programmers in
Seattle and Woods Hole are putting together the additional modules
necessary to allow LAS to work with profile data.
J-LAS will allow scientists to navigate and
visualize the merged data products in a much more convenient and rapid
collection of merged data products
is available via the U.S. JGOFS customized Live Access Server (LAS)
This document is derived from an article in U.S. JGOFS Newsletter,
volume 11 number 3 (November, 2001) by David M. Glover and Cynthia Chandler (Woods Hole Oceanographic Institution).