LAS Installation and Configuration

Version 5.0

Introduction

The LAS (Live Access Server) is an easily installed, configurable Web application especially suited to the analysis and visualization of large, gridded environmental data sets. It uses the Ferret program for data analysis and visualization, but other visualization tools can be used.

The Live Access Server enables the Web user to:

Architecture

LAS is really two separate servers -- a user interface server and a data server. Although both servers usually reside on the same computer, they can be distributed to different machines if desired. The user interface server handles the presentation logic for LAS. It consists of a set of Perl objects at the server, as well as static and dynamically generated HTML, JavaScript, and Java code that are downloaded to the user's Web browser. The browser code allows a user to select a dataset and variable to visualize, as well as geographical regions, view planes, visualization styles, and a variety of other parameters for given variable.

Once a user decides how he wants a given variable in a given dataset to be presented (as data or visualized), the browser sends a request (composed in XML) to the data server. The data server consists of a set of Perl objects that parse the XML request and call a configured driver for data analysis and visualization. LAS is currently configured to use Ferret as the data analysis and visualization program, but other programs can be substituted. The Ferret driver dynamically creates custom Ferret scripts which are then used by Ferret to visualize or analyze the requested data. The datasets can be on the local machine, or can be accessed remotely through the DODS protocol. The visualized or analyzed data is returned to the user in a variety of formats (as a GIF image, netCDF, ArcView GIS, comma separated values, or plain ASCII).

Both the user interface and data servers are configured with one XML configuration file. XML (the Extensible Markup Language) is a markup language that allows a user to store structured data in a text file. It is similar to HTML, but has the advantage of being extensible (through user defined tags) and easier for computer programs to use. A LAS administrator can easily change the configuration of the server by editing this configuration file. After the XML file is edited, the administrator runs a program that generates JavaScript for the user interface, and converts the XML file to a set of relational database tables that are utilized by both the data and user interface servers.

Downloading LAS

The latest released version of LAS is available from the LAS download page.

You can also pick up the latest development snapshot of LAS by using anonymous cvs. Here's how (on Unix platforms):

Installation

Requirements

You must have the following software:

Installation

The LAS configure script should be run as root, since it has to change file ownership. You can still run it as a non privileged user, but you will have to change file ownership and protection by hand.
  1. Pick a directory where you want the LAS files to go (referred to in this document as $lasroot), cd to that directory and untar the files:

    gunzip -c las3.tar.gz | tar xvf -

  2. If your Ferret environment is not set up, you should source the Ferret startup script. If Ferret is installed in /usr/local/ferret, then type:

    source /usr/local/ferret/ferret_paths

  3. Change to the lasxml directory and run the configure script:

    cd lasxml
    ./configure

    LAS can be configured to allow comparisons between variables (differencing, overlay plots). This is turned off by default. If you want to enable comparisons run the configure script as:

    ./configure -compare

    The configure script will ask you a number of questions, and make some tests to make sure you have all of the required software. Unless you plan on supporting multiple LAS servers on one Web server, or want to customise LAS code, the default answers should suffice.

  4. Change your Web server configuration file. If you are running Apache, the script will tell you which commands have to be added to the server config file. If not, the script will give you more general configuration instructions. Add the new commands to the server configuration file, restart the server, and you should be ready to test the sample LAS configuration.
    If you used the default configuration, and your Web server is on foo.com, just point your browser to:

    http://foo.com/las/

    to run LAS.

Adding datasets

Almost all of the configuration information used by LAS is stored in a XML file in $lasroot/lasxml/server/las.xml.

If your data is in COARDS compliant netCDF format, you can automatically add datasets to this configuration file. If you want to do this for a netCDF file named /a/foo.cdf, you can:

The first command adds the metadata from /a/foo.cdf to the XML configuration file; the second automatically generates JavaScript for the user interface server.

You will have to regenerate the user interface and update the relational database each time you modify the XML configuration file. If you don't use XML includes, you can use make. If you use XML includes, or use more than one XML configuration file, you will have to run genLas2.pl by hand:

If you now reload the LAS page (using Shift-Reload), the new dataset will appear in the left hand frame.

Note: you should always use Shift-Reload to reload the LAS home page after regenerating JavaScript and HTML. JavaScript errors that occur when accessing LAS are almost always due to automatically generated JavaScript being out of synch with LAS HTML. Shift-Reload forces your browser to clear its cache and use the latest version of JavaScript and HTML supplied by the Web server.

Debugging LAS

If you've diligently followed the installation instructions and LAS doesn't work, here's what you can do:

Understanding/modifying the XML configuration file

Automatic XML generation from netCDF files will only work if the netCDF files are COARDS compliant. If you wish to use LAS with other netCDF files, or if the metadata in the netCDF files is inadequate, you will have to learn how to modify the XML configuration file.

A sample XML file is below, followed by definitions for LAS XML elements and attributes.

Sample XML file

<?xml version='1.0' ?>
<!-- Associate the name StdOperations with the string "operations.xml" -->
<!DOCTYPE spec SYSTEM "spec.dtd" [
<!ENTITY StdOperations SYSTEM "operations.xml">
]>

<lasdata>
  <institution name="Pacific Marine Environmental Lab"
   url="http://www.pmel.noaa.gov"/>
<!-- Include operations. All standard LAS operations are in "operations.xml" -->
   <operations url="http://foo.server.com/las-bin/LASserver.pl">
        &StdOperations;
   </operations>

<!-- Define datasets/variables -->
   <datasets>
    <coads_climatology_cdf name="COADS Climatology" url="file:coads_climatology" doc="doc/coads_climatology.html">
     <variables>
      <airt name="Air Temperature" units="DEG C">
       <link match="/lasdata/grids/coads_climatology_cdf_grid"/>
      </airt>
      <sst name="Sea Surface Temperature" units="Deg C">
       <link match="/lasdata/grids/coads_climatology_cdf_grid"/>
      </sst>
     </variables>
     <composite>
      <wind_vectors name="Wind vectors" js="VecVariable">
       <link match="../../variables/uwnd"/>
       <link match="../../variables/vwnd"/>
      </wind_vectors>
     </composite>
    </coads_climatology_cdf>
   </datasets>
<!-- Define grids -->
   <grids>
    <coads_climatology_cdf_grid>
     <link match="/lasdata/axes/coads_climatology_cdf_COADSX"/>
     <link match="/lasdata/axes/coads_climatology_cdf_COADSY"/>
     <link match="/lasdata/axes/coads_climatology_cdf_TIME"/>
    </coads_climatology_cdf_grid>
   </grids>
<!-- Define axes -->
   <axes>
    <coads_climatology_cdf_COADSX type="x" units="degrees_east">
     <arange start="21" step="2" size="180"/>
    </coads_climatology_cdf_COADSX>
    <coads_climatology_cdf_COADSY type="y" units="degrees_north">
     <arange start="-89" step="2" size="90"/>
    </coads_climatology_cdf_COADSY>
    <coads_climatology_cdf_TIME type="t" units="month">
     <arange start="1-1-16" step="1" size="12"/>
    </coads_climatology_cdf_TIME>
   </axes>
  </lasdata>

XML Element and Attribute Definitions

The elements and attributes that make up a LAS XML specification are defined below. Note the following:

Lasdata element

The lasdata element is the root of the LAS XML hierarchy.

Element Tag Required? Children Description
lasdata <lasdata> yes datasets+ |
grids+ |
axes+ |
operations+ |
properties*|
institution?		 Root of LAS XML document.


Link element

The link element is used to associate one part of the XML specification with another. It is analogous to the anchor tag in HTML, with the exception that the contents that are linked to are substituted in place. The following XML:

<a>
  <b>
    <link match="/a/d"/>
  </b>
  <d name="foo"/>
</a>


is equivalent to:

<a>
  <b>
    <d name="foo"/>
  </b>
  <d name="foo/>
</a>



Element Tag Required? Children Description
link <link> no Links to XML content in another part of the document.
Attribute Type Required? Description
match string yes The path to the linked content. This can be either a full or relative path. If there are multiple matches, the first match is used.

Datasets element

The datasets section contains information on all of the datasets supported by the server. Here you will find metadata such as variable names, variable units, or the file or URL to be associated with a variable or dataset.

Element Tag Required? Children Description
datasets <datasets> yes dataset* Container for dataset tags


Dataset element

Each dataset is described by a dataset element. The name of the element can be anything but properties or institution; the name must be unique. It is coads_climatology_cdf in the sample file.

Element Tag Required? Children Description
dataset <*> no variables+ |
composite* |
properties |
institution		 Describes a dataset
Attribute Type Required? Description
name string no Display name of dataset. Defaults to element name.
url string no URL of the dataset. In most cases, the URL should use a file scheme; a http scheme will only work with DODS datasets. If not defined, then each variable element contained by this element must have a URL attribute defined.
doc string no URL of documentation associated with dataset. This documentation will appear when a user clicks on dataset in the LAS datasets frame.


Variables element

Element Tag Required? Children Description
variables <variables> yes variable* Container for variable tags

Composite element

Composite elements are used to describe "virtual" variables that are composed of one or more dataset variables.

Element Tag Required? Children Description
composite <composite> no comp_tags+ Container for composite tags


Comp_tag element

Element Tag Required? Children Description
comp_tag <*> yes variable+ Describes a virtual variable
Attribute Type Required? Description
name string yes Display name of composite element
js string yes JavaScript class to be associated with the composite. This class should be a subclass of the V JavaScript class, and should override the getOpType method (which determines the LAS operation to be associated with a variable). The VecVariable class in $lasroot/lasxml/ui/las.js illustrates how this should be implemented.

Variable element

Element Tag Required? Children Description
variable <*> no grid+ |
properties |
institution		 Describes a variable
js string yes JavaScript class to be associated with the variable. This class should be a subclass of the V JavaScript class, and should override the getOpType method (which determines the LAS operation to be associated with a variable). The VecVariable class in $lasroot/lasxml/ui/las.js illustrates how this should be implemented.


Grids element

Element Tag Required? Children Description
grids <grids> yes grid Container for grid tags



Grid element

Element Tag Required? Children Description
grid <*> no axis* Describes a grid


Axes element

Element Tag Required? Children Description
axes <axes> yes axis* Container for axis tags



Axis element

Element Tag Required? Children Description
axis <*> no arange | v+ Describes an axis
Attribute Type Required? Description
type 'x'|'y'|'z'|'t' yes Orientation of axis in space. 'x' implies a longitude axis, 'y' latitude, 'z' depth or height, and 't' time.
units string yes Axis units. Should be compatible with Unidata's udunits package.
display 'first'|'last' no Only valid for time axes. Default is 'first'. If 'first', the user interface will display the first time value for the axis when the dataset is initialized. If 'last', the last value for the axis will be displayed.



Arange element

Element Tag Required? Children Description
arange <arange> no Describes a regular axis range.
Attribute Type Required? Description
start double|time yes

Origin of the axis. If the axis is of type 't', this is a time string; for all other axes, this is a double precision floating point number.

The time string is of the form 'YYYY-MM-DD HH'. The year '0001' is reserved for climatological time axes.

step double yes The step size of this regular axis. The units used are those of the containing axis. The only time units supported are 'year', 'month', 'day', and 'hour'
size integer yes Number of points in the axis



V element

Element Tag Required? Children Description
v <v> no text

Describes a point in an irregular axis. For example, an axis with two values, 0 and 20, would contain the XML:
<v>0</v>
<v>20</v>

For time axes only, the values can be either time values (YYYY-MM_DD HH), integers, or strings. If the value can be interpreted as a time value, it is assumed to be a time value; if not, the UI generation program attempts to interpret the value as a integer; if it isn't a integer it is assumed to be a string. For integers, LAS will assume that the value of the integer is an index rather than a time value. For strings, LAS passes the position of the string in the time axis array as an index to the LAS data server.


Properties element

Properties are general purpose tags that can be associated with a LAS XML element. LAS currently uses properties to associate visualization "hints" for Ferret with datasets or variables. A set of Ferret properties might look like:

<properties>
  <ferret>
    <size>0.25</size>
    <format>netCDF</format>
  </ferret>
</properties>


The child elements of the <properties> tag can be anything; it is up to the server application to decode their semantics.

Element Tag Required? Children Description
properties <properties> no property* Container for property tags


Institution element


Element Tag Required? Children Description
institution <institution> no Describes the institution that supplied the dataset or variable. This information is displayed on the client's Web browser when a given dataset or variable is selected.
Attribute Type Required? Description
name string yes Display name of the institution.
url string no URL of Web page containing info on the institution.


Operations

Element Tag Required? Children Description
operations <operations> yes operation+ Container for operation tags
Attribute Type Required? Description
url string yes URL of LAS server implementing operations


Operation

An operation is used to map a browser user interface action to a Perl subroutine or method on a LAS server. For instance, there is one operation defined in the sample XML file above; this operation maps the UI action shade to the subroutine draw in the Perl package LAS::Server::Ferret.

Element Tag Required? Children Description
operation <*> yes arg+ Describes the mapping between client-side, JavaScript based user actions to server side, Perl based subroutines.
Attribute Type Required? Description
name string yes Description of operation. Solely for documentation.
class string yes Perl object or package to be associated with the operation.
method string yes Perl subroutine to be associated with the operation.

Arg

Element Tag Required? Children Description
arg <arg> yes Defines the type of an argument to be passed to a LAS operation. Not yet implemented.
Attribute Type Required? Description
type string yes Type of argument. Currently ignored.


Extending LAS

Overview

LAS can be customized in a number of different ways. Some of the available methods are (in order of increasing difficulty and increasing power and flexibility):

You can also plug in another visualization program as a back end. Prototype versions have been created for IDL, GrADS, and the NCL command language .

Ferret properties

Ferret data visualizations are customized by adding properties to the LAS XML configuration file. Properties can be specified for a given dataset, variable, or LAS server. For example, suppose you want 2D visualizations of all variables in the COADS climatology dataset to use the Plot Plus fill command. Add the following to the sample XML example (new XML is in red):

<coads_climatology_cdf name="COADS Climatology" url="file:coads_climatology" doc="doc/coads_climatology.html">
   
   <properties>
        <ferret>
            <fill_type>fill</fill_type>
        </ferret>
   <properties>
...

If you want this to apply to every dataset, you can add this at the top level of the XML hierarchy:

<lasdata>
   <properties>
        <ferret>
            <fill_type>fill</fill_type>
        </ferret>
   <properties>
...

If you only want this to apply to a variable you can add the <properties> tag to the variable rather than the dataset:

    <coads_climatology_cdf name="COADS Climatology" url="file:coads_climatology" doc="doc/coads_climatology.html">
     <variables>
      <airt name="Air Temperature" units="DEG C">
        <properties>
          <ferret>
              <fill_type>fill</fill_type>
          </ferret>
        <properties>

Variable level properties override dataset level properties which override file level properties.

Following are the currently supported Ferret properties:

Name Values Description
contour_levels Contour levels to use. Format is available in Ferret documentation.
 fill_type  fill|shade  Type of 2D fill algorithm to use. Fill uses a contour fill algorithm and shade uses a rectangular raster algorithm. Shade is faster.
 fill_levels    Fill levels to use. Syntax for fill levels is available in Ferret documentation.
 palette    Palette to use for color fills. A list of available palettes can be obtained by using the ferret Fpalette command.
 land_type  none|contour|shade  Type of continent fill to use when drawing land boundaries. None implies that no boundaries will be drawn, contour implies that only land outlines will be drawn, and shade implies that continents will be filled.
 init_script valid Ferret script file File name of Ferret script to be executed before processing a LAS operation. The file must be located in the $lasroot/server/jnls directory. See below for more details.
 script_prefix  character string Prepend the string to the name of the Ferret scripts to execute when a LAS operation is executed. For instance, a script_prefix of foo would cause LAS to look for foo_std_gif.tmpl rather than std_gif.tmpl.

Ferret initialization scripts

LAS can be configured to run a special Ferret initialization script for a given dataset or variable. You can use this to generate virtual variables -- variables which are dynamically generated by Ferret from variables that already exist in a dataset.

Suppose you have a dataset that contains variables representing air temperature and sea surface temperature, and that the names of the variables are, respectively, airt and sst. You would like LAS to also have a variable that represents the difference between the air temperature and the sea surface temperature. You can create this variable by using the <init_script> property. Here's how:

If you reload the LAS user interface in your browser (using Shift-Reload), you will now see a new variable named Air-Sea Difference.

Custom Ferret scripts

If you want greater control over customization of the Ferret backend than that provided by Ferret initialization scripts, you may want to create a set of custom Ferret scripts by using the <script_prefix> tag. The syntax is similar to the <init_script> tag:

    <properties>
      <ferret>
       <script_prefix>my_script_</script_prefix>
      </ferret>
    </properties>

Before creating a custom script, you need to understand how the LAS data server processes a data request. Every request contains a named operation. The named operation is mapped (through a set of operation tags specified in the XML configuration file) to a procedure in a Perl object or package. For instance, the sample XML file shipped with LAS includes the file operations.xml which contains the following XML fragment:

<shade name="Shade a variable" class="LAS::Server::Ferret" method="draw">

If LAS receives a data request specifying the shade operation, the draw procedure in the Perl package LAS::Server::Ferret will be called (if you want to see what this code looks like, look in $lasroot/server/Ferret.pl ). This code looks for a Ferret script named std_gif.

It is also important to understand that the Ferret driver for LAS looks for either templated or regular Ferret journal files. Templated Ferret journal files are Ferret journal files that have been enhanced with templates based on the Perl Template module (full documentation on the template module is available by typing perldoc Template). The template additions are used to overcome limitations in the Ferret scripting language. Templated files have a .tmpl suffix, while regular Ferret journal files have a .jnl suffix. An example of a templated Ferret journal file is available in $lasroot/server/jnls/std_gif.tmpl.

LAS uses the following algorithm to find a script:

foreach directory in (<custom directory> <$lasroot/server/jnls>){
	- Look for template file with <script_prefix> prepended
	- Look for journal file with <script_prefix> prepended
	- Look for template file without script prefix
	- Look for journal file without script prefix
	- Execute script if found, and return
}

For the draw operation, the script prefix my_script_, and custom code directories named custom, LAS will look for files named:

and will execute the first file that it finds.

The table below shows some of the Ferret scripts that are used by LAS:

Operation
Perl Method
Ferret scripts
shade
LAS::Server::Ferret::draw
std_gif
std_refmap
line
LAS::Server::Ferret::draw
std_gif
std_refmap
data
LAS::Server::Ferret::data
std_list
vector
LAS::Server::Ferret::vector
std_gif_vectorref
std_refmap

and the following table shows the arguments that are currently passed to templated Ferret scripts:

Variable Values Definition Default
contour_levels   Levels to use in a contour plot. Uses standard Ferret syntax for levels. null
dataset_name   Name of the current active dataset. Can be either a DODS dataset URL or a file name.  
do_contour null|0|1 Non-zero if a contour 2D plot should be drawn. null
do_shade null|0|1 Non-zero if a shaded 2D plot should be drawn. null
draw_reference null|0|1 Non-zero if a reference map should be drawn. null
fill_levels   Levels to use in a fill or shade plot. Uses standard Ferret syntax for levels. null
fill_type 'fill' | 'shade' Algorithm to use for a fill or shade plot. null
magnify -1|0|1|2|3 Hint on what kind of background map to draw. -1 is no map at all, 0 is low-resolution unfilled, and 1..3 are progressively higher resolution filled map. This parameter might change in the future.  
overlay_labels Perl list of strings Strings to use for plot labeling. null
overlay_variable_name   Name of variable to use for overlay plots  
palette   Name of Ferret palette to use 'default'
rank 1|2|3|4 Rank of variable.  
title   Plot title. null
variable_name   Name of current variable  
view x|y|z|t|
xy|xz|xt|yz|yt|zt|
xyz|xyt|yzt
 Names of axes that are contained in the current data view  

 

Customizing LAS code

Warning: The following LAS customization examples are experimental and might not function with new versions of LAS.

Example: adding an operation

You can customize LAS by adding code that supports a new operation. This is appropriate when you want LAS to return an entirely new type of visualization or data format for a particular variable. Here's an example of a new operation that creates a Ferret test plot:

Reload the LAS user interface and click on the Select Product menu. You will see a menu item named Test Plot (GIF) at the bottom of the menu. Select this item and click on the Get Data button. You should see a Ferret "test pattern" plot.

Example: mapping a time to a filename

Datasets are frequently distributed as a collection of files rather than a single file. Each file might represent a particular time point, and the file might be named such that the time point is represented by the file name. Suppose we have a series of files that represent data from one day, and each file is named as YYYYMMDD.cdf. The file 20000220.cdf would contain data from Mar 20, 2000.

A separate variable would have to be defined for each time point if this data were represented in LAS as shipped. This can be avoided with a few LAS code modifications.

Note that LAS will be unable to make any plots of data along a time axis. The user interface will be modified to reflect this.

The following example assumes that the time range of the data is from Jan 1, 2000 to Dec 31, 2000, and that each file has one variable named SST on a longitude/latitude grid of 1 degree.

Adding a new visualization program

The following example shows how to add a new visualization program to LAS. First, we will create a new test operation for a variable in a dataset, and that test operation will run a small Perl script that writes data to a text file that will be sent back to the browser.

You can use the CLC LAS Perl module to run the visualization program. The program should be command line driven -- there must be a prompt string that the CLC module can search for to determine if the program has completed a particular operation.

This example uses the sample Perl script in $lasroot/server/clctest.pl. It is a simple command line program that has two functions:

Here's how to run this example:

   <datasets>
    <test_server name="Test server" url="file:foo" js="TestServer">
      <variables>
         <test name="Test">
             <link match="/lasdata/grids/coads_climatology_cdf_grid"/>
         </test>
      </variables>
    </test_server>

Comments to: las@pmel.noaa.gov
Last Modified: February 9, 2001