How to place data on your own JGOFS Server

or, I'd prefer to serve it from my own machine, thank you.

_____________________

Introduction

The JGOFS Data system was designed with the flexibility to serve data from the Investigator's own workstation. To date, much of the data that is served by the U.S. JGOFS program has been submitted to the Data Management Office (DMO) and is served by its Sun workstation. However, to encourage JGOFS Investigators to serve their data from their own Unix workstation, we have prepared this guide.

Determine if your format can be read by the system

After data is ready to be shared with other Investigators in the program, check its format against the samples provided by the DMO. If the format of your data does not exactly match one of the existing sample formats, a decision needs to be made to either change the data format to conform to the most similar sample format or to request a modification to one of the formats. If the data does match one of the formats, note the name of the "method" or translation program that can read that format. Now you know most of the pieces required by our system: the method name, the name(s) of the files to be served, and where they will reside on your system.

Install a JGOFS server

The next step is to download the system software from the Distribution site. Follow the instructions offered with the software for downloading the JGOFS software as a tar file. We suggest that a new user be created on your system with the username, 'jgofs'. This user can be the one who installs and, therefore, owns the software. The user 'jgofs' should have its own group id and a user id that differs from any other user on your system. Install the JGOFS Data System according to the directions online. If you have problems while installing the software, contact the DMO for assistance.

Placing data on the server

Once you have a working server installed on your Unix workstation, you are ready to tell it where your data are located. As user 'jgofs', make certain that you can read the data files [suggestion: 'more' one or more of the files]. This will ascertain that the system (run as jgofs) will have permission to read and serve the data.

There are two files that tell the JGOFS system where to find data and how to read it:

.remoteobjects
.objects

The .remoteobjects file contains a three line entry for every object available through the JGOFS data system in a given category. The three lines entered in the .remoteobjects file should be inserted alphabetically by the Principal Investigator's last name and will appear in this order on the system. The information given by the 3 line entry is:
1. the name of the data object to display, limited to 18 characters
  the machine where the data is located, even if that machine is the local machine
  the location of the .objects file on the serving machine
  the name of the data object entry in the .objects file.
2. the name of the Principal Investigator responsible for collecting the data
3. a brief description of the contents of the data object
The structure of the 3-line entries in .remoteobjects is, where ^ indicates a required blank space:

display_object=//machine.domain/category-subdirectory-path/objname
-^P.Investigator
-^Brief description of the data

An example of an entry is:

hydro=//usjgofs.whoi.edu/jgofs/eqpac/tt007/hydro
- J.Murray
- Basic hydrographic measurements
The .objects file contains a one (1) line entry for each object that it serves. The object name in the .objects file must exactly match the object name at the end of the URL-like entry beginning with //machine in the corresponding .remoteobjects file.
This one line entry provides the following information:

the exact directory and filename(s) of the data on the local system
which method, or translation program, to use to access it
optionally, limits the parameters displayed by the system

Structure of .objects:
objname=method(/exact-absolute-path-on-local-system/filename)
An example of an entry is:
hydro=def(/data/eqpac/murray/hydro_t7.def)
Optionally, one may restrict the parameters to be served. For example, an extended syntax allows:
objname=method(/path/file[,optional_list_parameters])
The existence of the item [,optional_list_parameters] indicated above, identifies the parameters which will be served by the system and may restrict which of these will be served, also. For example, a line such as
ctd=def(/data/nabe/a119.ctd,sta,press,sal,par,sta<150)
displays only station, pressure, salinity, and PAR parameters, excluding any others which may reside in the data file a119.ctd. A further restriction to the parameters is made by specifying only stations numbered less than 150 e.g. sta<150.
Inquire of the DMO staff for correct syntax for your object, if you are unsure.

These two files are located in the objects subdirectory tree of the JGOFS Data System. For example, if you chose to install the system at /home/machinename/jgofs, your data objects information will be located starting at /home/machinename/jgofs/objects. Below this level are the directories which can be displayed by the system.

The main U.S. JGOFS server has as its top level, a /jgofs/ category, below which are the process study categories. We list /jgofs/eqpac/tt007 as the category of data from the Equatorial Pacific process study, cruise tt007.

Assume that you are serving data from the Arabian Sea, cruise ttn-042. You would create a subdirectory structure like this: /home/machinename/jgofs/objects/jgofs/arabian/ttn-042 indicating that you want to display /jgofs/arabian/ttn-042 as your category of data. The .remoteobjects and .objects files containing the information about the data for ttn-042 would be located in this directory.

Documentation

Documents detailing protocols of scientific methodology related to sample or data collection, need to be available for proper utilization of the data. We ask that these documents be submitted to the DMO before you begin to serve your data. Notes specific to a particular dataset, perhaps indicating anomalies at the time of collection or during the processing of data can be served with the information, documenting the parameter names used in the data. We do require that the parameter names be defined and that the units be clearly stated. The name of the object (from the .remoteobjects file) is used to provide the filename for another file that is also part of the system.

The objectname.info file is located in the same directory as the .remoteobjects and .objects files. It provides information related to data collection as well as the object's parameter names, their descriptions and units. Also included are the ship name, cruise identifier, the Principal Investigator(s)'s full name and links to any additional notes. Place four spaces between the longest parameter name and its description and keep lines to an 80 character maximum length to ensure the list will be visually aesthetic. A template for the objectname.info file is provided.

Spot-check the data

All the pieces required to display data on the system are now in place. Using a browser, visually inspect the data to insure proper alignment, etc. Use the "List all at next level" followed by the "Flat listing" features of the system and refer to the original file to insure that every record of the dataset is displayed.

Let the world know

The data is ready to be incorporated into the category collection. In other words, the data that you have prepared and are now serving needs to be made available to the other Investigators as part of the Process Study data set. To do this, send the .remoteobjects file to the Data Management Office to be added to its larger file of entries. The DMO staff will update the List of Available Objects.

Congratulations!

_____________________

To the Data Management Page