Personal tools

Operational Modelling

From MohidWiki

Jump to: navigation, search

Overview

This wiki intends to explain how operational models are set in the Mohid Maretec framework.

Before intending to put any model in operational, the model has to be strongly tested.

Prerequisites

Mohid and Matlab have to be installed. Mohid version and dll needed in case of being 32 or 64 bits.

Folders Needed

In the folder where the model is going to be set up, it is needed to define the following generic folders:

  • Backup
  • GeneralData
  • Logs
  • Plots
  • Work (only if atmospheric interpolation is needed)

Also a folder for the first level model will be defined, and it will contain the following subfolders:

  • data
  • exe
  • figures
  • res
  • a folder with the name of the submodel in case it exist, this folder must replicate this same estructure

GeneralData folder

This folder must contains the following subfolders:

  • Bathymetry: Here it should be contained the bathymetries used by the model
  • BoundaryConditions: Folders for the boundary conditions should be defined here (Atmosphere, Hydrodynamics, etc.). In them, the temporary files for each run should be copied.
  • Exe: In this folder should be located the MohidWater.exe file, this is the only name that is accepted for the Mohid executable file.

Also should contain any other subfolders needed for input/output definitions as discharges, timeseries or boxes files

Input Data

For each of the different modelling levels, a set of typical input data for each of the modules have to be selected (model.dat, hydrodynamic.dat, waterproperties.dat, turbulence.dat, etc.) also a set of hdf5 final files (.fin) for the hydrodynamic, GOTM and waterproperties in case they exist.

Atmospheric forcing

If the model have interpolated atmospheric forcing data for the particular gridded data, a location should be provided. In the case that it need to be calculated, a folder named work should be included with the configuration files and programs. Atmospherical forcing conditions can be provided by different sources: GFS, MM5 and WRF.

MM5

Two consecutive programs would be executed here, a first program GetWindForecast.exe that will go to find the files corresponding for the selected dates and that automatically will glue the files using a second software ConvertToHDF5Action.exe. The input file for the first software is described below.

GetMM5Forecast.dat
MM5_PATH                     : \\ftpserver\ftp.mohid.com\LocalUser\meteoIST\mm5_6h\
WORK_PATH                    : D:\Aplica\PCOMS\Work\MM5\
OUTPUT_FOLDER                : D:\Aplica\PCOMS\Work\MM5\History\
BACKUP_FOLDER                : \\datacenter\alexandria4\modelos\Mohid_IST\PCOMS\Portugal\Meteo\
MOHID_GRID                   : Portugal.dat
MAIL_SERVER                  : mail.ist.utl.pt
MAIL_SENDER_NAME             : Portuguese Coast Operational Modelling System
MAIL_SENDER_ADDRESS          : 
CHECK_NUMBER_OF_FILES        : 1
NUMBER_OF_FILES              : 13
<begin_out_email_list>
<end_out_email_list>

Once the first software has performed the find and glue task, the ConvertToHDF5Action.exe will generate a input file for the desired period of time and will interpolate for the selected grid.

ConvertToHDF5Action.dat
<begin_file>
ACTION                        : PATCH HDF5 FILES
TYPE_OF_INTERPOLATION         :  3
START                         : 2011 4 19 0 0 0
END                           : 2011 4 22 0 0 0
<<begin_father>>
FATHER_FILENAME               : MM5_D2_Portugal.hdf5
FATHER_GRID_FILENAME          : D2.dat
LEVEL                         :  2
<<end_father>>
<<begin_father>>
FATHER_FILENAME               : MM5_D3_Portugal.hdf5
FATHER_GRID_FILENAME          : D3.dat
LEVEL                         :  3
<<end_father>>
OUTPUTFILENAME                : MM5_Portugal_2011-04-19_2011-04-22.hdf5
NEW_GRID_FILENAME             : Portugal.dat
<<BeginFields>>
air temperature
downward long wave radiation
wind velocity X
wind velocity Y
wind modulus
relative humidity
solar radiation
mean sea level pressure
<<EndFields>>
<end_file>

WRF

Two consecutive programs would be executed here, a first program GetMeteoForecast.exe that will go to find the files corresponding for the selected dates and that automatically will glue the files using a second software ConvertToHDF5Action.exe. The input file for the first software is described below.

GetMeteoForecast.dat
METEO_MODEL                              : WRF
METEO_MODEL_PATH                         : \\DATACENTER\alexandria3\modelos\Meteo_IST\wrf_6h
WORK_PATH                                : D:\Aplica\OPTagus3D\Work\WRF\
OUTPUT_FOLDER                            : D:\Aplica\OPTagus3D\Work\WRF\History\
BACKUP_FOLDER                            : \\datacenter\alexandria4\modelos\Mohid_IST\OPTagus3D\Tagus\Meteo\
MOHID_GRID                               : D:\Aplica\OPTagus3D\GeneralData\Bathymetry\TagusRibeiras.dat
MOHID_GRID_LABEL                         : Tagus3D
MAIL_SERVER                              : mail.ist.utl.pt
MAIL_SENDER_NAME                         : OPTagus3D
MAIL_SENDER_ADDRESS                      : 
!So por ser hindcast
START                                    : 2011 2 6 0 0 0
END                                      : 2011 2 7 0 0 0
!NUMBER_OF_DAYS                          : 1
!CHECK_NUMBER_OF_FILES                   : 1
!NUMBER_OF_FILES                         : 5
<begin_domain_list>
WRF_3km Lisbon 3
<end_domain_list>
<begin_out_email_list>
<end_out_email_list>

Once the first software has performed the find and glue task, the ConvertToHDF5Action.exe will generate a input file for the desired period of time and will interpolate for the selected grid.

ConvertToHDF5Action.dat
<begin_file>
ACTION                        : INTERPOLATE GRIDS
TYPE_OF_INTERPOLATION         :  3
START                         : 2011 2 6 0 0 0
END                           : 2011 2 7 0 0 0
<<begin_father>>
FATHER_FILENAME               : WRF_WRF_3km_Lisbon.hdf5
FATHER_GRID_FILENAME          : WRF_3km.dat
LEVEL                         : 3
<end_father>>
OUTPUTFILENAME                : WRF_Tagus3D_2011-02-06_2011-02-07.hdf5
NEW_GRID_FILENAME             : D:\Aplica\OPTagus3D\GeneralData\Bathymetry\TagusRibeiras.dat
<<BeginFields>>
air temperature
downward long wave radiation
wind velocity X
wind velocity Y
wind modulus
relative humidity
solar radiation
mean sea level pressure
<<EndFields>>
<end_file>

Running Mohid

In each of the exe folder a nomfich.dat should be placed and modified with the needed paths.

nomfich.dat
IN_BATIM                      : ..\..\GeneralData\Bathymetry\Portugal_Window.dat
ROOT                          : D:\Aplica\OPTagus3D\Portugal_Window\res\
ROOT_SRT                      : D:\Aplica\OPTagus3D\Portugal_Window\res\Run1\
IN_MODEL                      : D:\Aplica\OPTagus3D\Portugal_Window\data\Model_1.dat
SURF_DAT                      : D:\Aplica\OPTagus3D\Portugal_Window\data\Atmosphere_1.dat
SURF_HDF                      : D:\Aplica\OPTagus3D\Portugal_Window\res\Atmosphere_1.hdf
DOMAIN                        : D:\Aplica\OPTagus3D\Portugal_Window\data\Geometry_1.dat
AIRW_DAT                      : D:\Aplica\OPTagus3D\Portugal_Window\data\InterfaceWaterAir_1.dat
AIRW_HDF                      : D:\Aplica\OPTagus3D\Portugal_Window\res\InterfaceWaterAir_1.hdf
BOT_DAT                       : D:\Aplica\OPTagus3D\Portugal_Window\data\InterfaceSedimentWater_1.dat
BOT_HDF                       : D:\Aplica\OPTagus3D\Portugal_Window\res\InterfaceSedimentWater_1.hdf
IN_DAD3D                      : D:\Aplica\OPTagus3D\Portugal_Window\data\Hydrodynamic_1.dat
OUT_DESF                      : D:\Aplica\OPTagus3D\Portugal_Window\res\Hydrodynamic_1.hdf
IN_TURB                       : D:\Aplica\OPTagus3D\Portugal_Window\data\Turbulence_1.dat
TURB_HDF                      : D:\Aplica\OPTagus3D\Portugal_Window\res\Turbulence_1.hdf
DISPQUAL                      : D:\Aplica\OPTagus3D\Portugal_Window\data\WaterProperties_1.dat
EUL_HDF                       : D:\Aplica\OPTagus3D\Portugal_Window\res\WaterProperties_1.hdf
OUTWATCH                      : D:\Aplica\OPTagus3D\Portugal_Window\res\Outwatch_1.txt
DT_LOG                        : D:\Aplica\OPTagus3D\Portugal_Window\res\DTLog_1.txt

and also a Tree.dat file with the locations of the respective submodels to be run, in case there is a problem and only one of them is to be ran, it is enough to include an empty line between the different domains.

Tree.dat
Automatic Generated Tree File
by MOHID GUI
+D:\Aplica\OPTagus3d\Portugal_Window\exe
++D:\Aplica\OPTagus3d\Portugal_Window\3DTagusWQ\exe

Hindcasting

In this case it will be used the executable AutoTagusROFI.exe and the configuration file is TagusROFI-Auto.dat, in this file should be indicated the location of the specific configuration input files and the minimum and maximum time that the model should run, to stop the operational procedure in seconds.

TagusROFI-Auto.dat
START                      : 2011 01 08 0 0 0
END                        : 2011 03 01 0 0 0
TAGUSROFI_INPUT_FILE       : \\turbo\aplica\OPSAndre\TagusROFI.dat
TAGUSROFIPLOTS_INPUT_FILE  : \\turbo\aplica\OPSAndre\Plots\MatlabPlots.dat
TAGUSROFI_METEO_INPUT_FILE : \\turbo\aplica\OPSAndre\Work\WRF\GetMeteoForecast.dat
TAGUSROFI_BATCH            : \\turbo\aplica\OPSAndre\TagusROFI.bat
PLOTS_ONLY                 : 0
MIN_TIME                   : 200
MAX_TIME                   : 30000
TagusROFI.dat
MAIN_PATH                                : D:\Aplica\OPTagus3D\
MAIL_SERVER                              : mail.ist.utl.pt
MAIL_SENDER_NAME                         : OPTagus3D
MAIL_SENDER_ADDRESS                      : 
START                                    : 2011 2 12 0 0 0
END                                      : 2011 2 13 0 0 0
OPENMP                                   : 1
OPENMP_NUM_THREADS                       : 6
<begin_out_email_list>
List of e-mail addresses to be contacted in the format name <email address>
<end_out_email_list>
<begin_model>
NAME                                     : Portugal_Window
PATH                                     : Portugal_Window\
MODEL_HAS_OBC                            : 1
OBC_WORK_PATH                            :
OBC_SIMULATED_DAYS                       : 1 (By default 3)
MODEL_HAS_METEO                          : 0
GRID_FILE                                : 
MODEL_RUN_ID                             : 1
DT                                       : 180
BACKUP_PATH                              : D:\Aplica\OPTagus3D\BackUp\Portugal_Window\
STORAGE_PATH                             : \\DATACENTER\alexandria4\modelos\Mohid_IST\OPTagus3D\Portugal_Window\
MODEL_HAS_WATERPROPERTIES                : 1
MODEL_HAS_SURFACE_HDF                    : 0
MODEL_HAS_GOTM                           : 0
MODEL_HAS_OUTPUT_WINDOW                  : 0
MODEL_HAS_SOLUTION_FROM_FILE             : 1
<end_model>
<begin_model>
NAME                                     : 3DTagusWQ
PATH                                     : Portugal_Window\3DTagusWQ\
MODEL_HAS_OBC                            : 0
MODEL_HAS_METEO                          : 1
METEO_WORK_PATH                          : D:\Aplica\OPTagus3D\Work\WRF\History\
GRID_FILE                                : Portugal_Window\GeneralData\Bathymetry\TagusRibeiras.dat
MODEL_RUN_ID                             : 1
DT                                       : 12
BACKUP_PATH                              : D:\Aplica\OPTagus3D\BackUp\3DTagusWQ\
STORAGE_PATH                             : 
MODEL_HAS_WATERPROPERTIES                : 1
MODEL_HAS_INTERFACESEDIMENTWATER         : 1
MODEL_HAS_SURFACE_HDF                    : 1
MODEL_HAS_GOTM                           : 1
MODEL_HAS_OUTPUT_WINDOW                  : 0
<end_model>

Forecasting

In this case the executable to be used will be TagusROFI-OP.exe

Running Manually

The batch file to be used will be MOHID_run - manual.bat and should be edited according to the project, as an example:

cd "D:\Aplica\OPSAndre\Portugal_Window\exe"
D:
"D:\Aplica\OPSAndre\GeneralData\Exe\MohidWater.exe"
pause

Backup and Storage

A folder containing the name of each submodels should be created in the backup folder, i.e.

D:\Aplica\OPSAndre\BackUp\Portugal_Window\

and also a location in the dataserver should be created for the storage, i.e.

\\datacenter\alexandria4\modelos\Mohid_IST\OPSAndre\Portugal_Window\

Plots

It is needed to have matlab and m_map previously installed. The matlab can be found in the computer and the m_map can be downloaded from http://www.eos.ubc.ca/~rich/map.html and once unzipped it should be placed in the toolbox folder where the Matlab has been installed.

To install M_Map as a toolbox accessible to all users this, unpack the files into 
$MATLAB/toolbox/m_map, 
add that directory to the list defined in $MATLAB/toolbox/local/pathdef.m,
matlabroot,'\toolbox\m_map;', ...
and update the cache file using
rehash toolboxcache
You should restart before this take effect.

In order to plot automatically the mohid results, it will be needed the software MatlabPlot.exe that is configured by the file MatlabPlots.dat and executed by the file make_plots.bat.

make_plots.bat
matlab -nodesktop -nosplash -r mat_main_plot -minimize -wait
MatlabPlots.dat
WORK_PATH                                : D:\aplica\OPTagus3D_2010\Plots\
MAIL_SERVER                              : mail.ist.utl.pt
MAIL_SENDER_NAME                         : OPTagus3D_2010 - Plots
MAIL_SENDER_ADDRESS                      : 
START                                    : 2011 2 14 0 0 0
END                                      : 2011 2 15 0 0 0
<begin_out_email_list>
name <email address>
<end_out_email_list>
SUCCESS_FILE                             : D:\aplica\OPTagus3D_2010\MOHID_SUCCESS.dat
MOHID_PATH                               : D:\aplica\OPTagus3D_2010\BackUp\3DTagusWQ\Results_HDF\
PUBLISH_PATH                             : D:\aplica\OPTagus3D_2010\Plots\Publish\
PLOT_TEMP_SALT_VEL                       : 1
PLOT_BIO                                 : 1
BIO_MODEL                                : waterquality
ADD_DAYS                                 : 1

Also some matlab files have to be edited and modified for each application,

mat_setup_plots.m
mat_setup_dir = 'D:\Aplica\OPTagus3D_2010\Plots';
mat_config_coastline.m
coastline_file        = 'tagus_coastlineh.mat';
save_coastline        = false;          
coastline_res         = 'i';        %'l'=low, 'i'=intermediate, 'h'=high, 'f'=full;  

To set the limits and the colour scale of each property for instance,

mat_config_contour.m
sst_contourlimits     = [12 26];
sst_contour_scale     = [12 13 14 15 16 17 18 19 20 21 22 23 24 25 26];


Validation

to validate with OSTIA SST images first it should be installed the mexcdf package for matlab that allows operations with netcdf.

matlabroot,'\toolbox\mexcdf\mexnc;', ... matlabroot,'\toolbox\mexcdf\snctools;', ...

Sections

Variable List
1 =  temperature'
2 =  salinity'
3 =  density'
4 =  Density - SigmaT'
_______________________________'
Life Model'
_______________________________'
7 =  diatoms chlorophyll'
8 =  autotrophic flagellates chlorophyll'
9 =  nitrate'
10 = silicate acid'
11 = inorganic phosphorus'
12 = oxygen'
13 = diatoms carbon'
14 = autotrophic flagellates carbon'
15 = semi-labile dissolved organic carbon'
16 = labile dissolved organic carbon'
17 = particulate organic carbon'
18 = microzooplankton carbon'
19 = heterotrophic bacteria carbon'
20 = biogenic silica'
26 = short wave solar radiation'
27 = clorofila_total (diatoms + flagellates)'
28 = carbono_total (diatoms + flagellates)'
_______________________________'
Hydrodynamic'
_______________________________'
22 = velocity U'
23 = velocity V'
24 = velocity W'
25 = velocity modulus'
_______________________________'
Energy'
_______________________________'
29 = power density'
_______________________________'
Water Quality Model'
_______________________________'
5 =  cohesive sediment'
6 =  phytoplankton'
12 = oxygen'
26 = short wave solar radiation'
30 = nitrate'
31 = nitrite'
32 = ammonia'
33 = dissolved non-refractory organic nitrogen'
34 = dissolved refractory organic nitrogen'
35 = particulate organic nitrogen'
36 = dissolved refractory organic phosphorus'
37 = particulate organic phosphorus'
38 = zooplankton'
_______________________________'
ATMOSPHERE Model'
_______________________________'
39 = wind velocity X'
40 = wind velocity Y'
41 = solar radiation'
42 = precipitation'
43 = atmospheric pressure'
44 = air temperature'
45 = relative humidity'
46 = cloud cover'
47 = mean sea level pressure'
48 = wind modulus'
49 = wind direction'
50 = specific humidity'
_______________________________'
Water air interface'
_______________________________'
51 = latent heat'
52 = sensible heat'
53 = evaporation'
54 = net long wave radiation'
55 = upward long wave radiation'
56 = downward long wave radiation'
57 = short wave solar radiation'
58 = long wave solar radiation'
59 = short wave solar radiation extinction'
60 = long wave solar radiation extinction'
61 = oxygen flux'
62 = wind shear velocity'
63 = surface radiation'
64 = wind stress X'
65 = wind stress Y'
66 = surface water flux'
67 = non solar flux'
69 = albedo'

Publishing online

Operational Models

PCOMS running in mar computer

BIOPCOMS WQ running in pcoms-bio computer

PCOMS-BIO2 Life running in pcoms-bio computer

OPTagus3D running in Tagus computer

OPTagus3D_2010 running in Turbo computer

To be done

- Include a delay time of 6 hours to test again

- To do independent forecast pcoms and pcoms-bio

- Harmonize tools