Personal tools

Thredds

From MohidWiki

Revision as of 12:23, 26 February 2009 by Luisfernandes (talk | contribs) (→‎data.mohid.com)
Jump to: navigation, search

Thredds (Thematic Realtime Environmental Distributed Data Services) is a tomcat servlet that serves xml catalogs of openDAP datasets.

Setup

A working tomcat server is a prerequisite. Simply copy the thredds.war file obtained from the thredds homepage into the tomcat webapps directory.

data.mohid.com

  • How do I restart the tomcat and thredds server in data.mohid.com?
> cd /home/guillaume/Software/las7/tomcat/tomcat/bin
> ./Tomcat5.sh stop
> ./Tomcat5.sh start
  • How do I know if tomcat is really running?:
> netstat -tlnap | grep 8080

If it's running then a port 8080 must appear in the results of the command.

VmMrtSrv06

  • 192.168.20.136 when it is resident on the LAN.
  • To configure the tomcat to start during boot:
> sudo vim /etc/rc.d/rc.local
rc.local> /home/user/Download/apache-tomcat-6.0.18/bin/startup.sh

Catalogs architecture

We propose the following architecture as a good practice for maintaining catalogs.

Directory tree structure

$TOMCAT_HOME/content/thredds

  • threddsConfig.xml: configuration xml file of the Thredds server.
  • catalog.xml: catalog file containing links to all the subcatalogs containing the netcdf data.
    • pcomsCatalog.xml: catalog file pointed by the main catalog.

threddsConfig.xml

Simply edit the file and enable the netcdf subset service. 

#threddsConfig.xml
 <NetcdfSubsetService>
   <allow>true</allow>
   <dir>/data/tmp/thredds/ncSubsetCache/</dir>
   <scour>15 min</scour>
   <maxAge>30 min</maxAge>
 </NetcdfSubsetService>

catalog.xml

#catalog.xml
<?xml version="1.0" encoding="UTF-8"?>
<catalog name="THREDDS Server Default Catalog : You must change this to fit your server!"
       xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0"
       xmlns:xlink="http://www.w3.org/1999/xlink">
  <catalogRef xlink:title="PCOMS Catalog" xlink:href="/opendap/pcoms/pcomsCatalog.xml" name=""/>
</catalog>

pcomsCatalog.xml

#pcomsCatalog.xml
<?xml version="1.0" encoding="UTF-8"?>
<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink"
  name="Unidata THREDDS-IDD NetCDF-OpenDAP Server" version="1.0.1">

 <service name="latest" serviceType="Resolver" base="" />
 <service name="all" serviceType="Compound" base="">
   <service name="ncdods" serviceType="OPENDAP" base="/thredds/dodsC/" />
   <service name="HTTPServer" serviceType="HTTPServer" base="/thredds/fileServer/" />
   <service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/grid/" />
 </service>

 <dataset name="PCOMS Model Data">
   <metadata inherited="true">
     <serviceName>all</serviceName>
     <authority>maretec.ist.utl.pt</authority>
     <dataType>Grid</dataType>
     <dataFormat>NetCDF</dataFormat>
     <documentation type="rights">Freely available</documentation>
     <documentation xlink:href="http://www.mohid.com/operational/" xlink:title="Portuguese Coast Operational Modeling System"></documentation>
     <creator>
       <name vocabulary="DIF">Maretec/IST</name>
       <contact url="http://www.mohid.com/" email="http://www.mohid.com/contacts.htm" />
     </creator>
     <publisher>
       <name vocabulary="DIF">Maretec/IST</name>
       <contact url="http://www.mohid.com/" email="http://www.mohid.com/contacts.htm" />
     </publisher>
     <timeCoverage>
       <start></start>
       <end>present</end>
     </timeCoverage>
   </metadata>

   <datasetScan name="PCOMS Data" ID="PcomsDatasetScan"
                path="pcoms" location="/opendap/pcoms/"
                harvest="true">
     <metadata inherited="true">
       <documentation type="summary">The PCOMS model consists of a downscaling of the Mercator-Ocean PSY3V1 north-atlantic solution for the Portuguese coast, where the fes2004 tide solution is added, and the surface forcing is performed using the atmospheric forecasts from the 9 km resolution MM5 model run at http://meteo.ist.utl.pt. The model is run in full baroclinic mode with an horizontal resolution of 6 km and with 43 layers in the vertical axis with finer resolution of 5 m near the surface.</documentation>
       <geospatialCoverage zpositive="down">
         <northsouth>
           <start>34.4</start>
           <size>10.6</size>
           <units>degrees_north</units>
         </northsouth>
         <eastwest>
           <start>-12.6</start>
           <size>7.1</size>
           <units>degrees_east</units>
         </eastwest>
         <updown>
           <start>0</start>
           <size>5500</size>
           <units>m</units>
         </updown>
       </geospatialCoverage>
       <variables vocabulary="CF-1.0" />
       <variables vocabulary="">
         <variable name="temperature" vocabulary_name="temperature" units="degC">temperature</variable>
         <variable name="salinity" vocabulary_name="salinity" units="1e-3">salinity</variable>
         <variable name="ssh" vocabulary_name="ssh" units="m">sea water level</variable>
         <variable name="u" vocabulary_name="u" units="m/s">east-west current velocity</variable>
         <variable name="v" vocabulary_name="v" units="m/s">north-south current velocity</variable>
         <variable name="vm" vocabulary_name="vm" units="m/s">current velocity modulus</variable>
       </variables>
     </metadata>

     <filter>
       <include wildcard="PCOMSV1_*.nc" />
     </filter>
     <addID/>
     <sort>
       <lexigraphicByName increasing="false"/>
     </sort>
     <addLatest/>
     <addDatasetSize/>
     <addTimeCoverage datasetNameMatchPattern="PCOMSV1_([0-9]{4})([0-9]{2})([0-9]{2}).*.nc$"
                      startTimeSubstitutionPattern="$1-$2-$3T00:00:00"
                      duration="3 days" />
   </datasetScan>
 </dataset>
</catalog>

Building thredds urls

Dataset

  • Viewing a ${catalog} file tree named ${catalog}.xml: 
http://data.mohid.com:8080/thredds/${catalog}.html
  • Viewing a specific ${dataset} thredds metadata:
http://data.mohid.com:8080/thredds/${catalog}.html?dataset=${dataset.id}
  • Viewing a specific ${dataset}:
http://data.mohid.com:8080/${dataset.service.base}/${dataset.urlpath}

DatasetScan

  • Viewing a particular ${datasetscan} tree from any ${catalog} in the thredds server:
http://data.mohid.com:8080/thredds/catalog/${datasetscan.path}/catalog.html
  • Viewing a specific ${datasetscan.filteredfilename} thredds metadata:
http://data.mohid.com:8080/thredds/catalog/${datasetscan.path}/catalog.html?dataset=${datasetscan.id}/${datasetscan.filteredfilename}.nc.html
  • Viewing a specific ${datasetscan.filteredfilename} dataset:
http://data.mohid.com:8080/${datasetscan.service.base}/${datasetscan.path}/${datasetscan.filteredfilename}.nc.html

A sample catalog file

[XML] 

<?xml version="1.0" encoding="UTF-8"?>

<catalog name="MOHID THREDDS Server Catalog"
       xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0"
       xmlns:xlink="http://www.w3.org/1999/xlink">

 <service name="MOHID DODS" serviceType="OpenDAP" base="/thredds/dodsC/" />
 <datasetRoot path="test" location="content/testdata/"/>

 <dataset name="Test Single Dataset" ID="testDataset" serviceName="MOHID DODS"
          urlPath="test/testData.nc"/>

 <datasetScan name="Test all files in a directory" ID="testDatasetScan"
              path="testAll" location="content/testdata">

   <metadata inherited="true">
     <serviceName>MOHID DODS</serviceName>
   </metadata>

   <filter>
     <include wildcard="*.nc"/>
   </filter>

 </datasetScan>

 <catalogRef xlink:title="Test Enhanced Catalog" xlink:href="enhancedCatalog.xml" name=""/>

</catalog>

See also

External references

Retrieved from "http://wiki.mohid.com/index.php?title=Thredds&oldid=1512"