13.5 Ensemble Verification




In the last few sections, several different ensembles were created to analyze the uncertainty in the air concentration calculation using the probablility analysis program conprob. However, if we are only concerned with the mean from multiple simulations, a procedure exists to compute only the mean of multiple concentration input files. In the previous section, we created five ensemble members and compared the results with measurements, but the process was a bit convoluted. There is a way to automate the statistical analysis. Create the five meteorological model members hysplit2.001 (narr), hysplit2.002 (era40), hysplit2.003 (gblr), hysplit2.004 (wrf27), and hysplit2.005 (wrf09) if they are not already available in the working directory.

  1. As before, it is necessary to insure that in the Setup Run / Grid Menu the concentration output file is just named hysplit2. The base name will then be passed through the GUI to the ensemble scripts, where the programs automatically search for the 3-digit suffix. We can then create an ensemble mean through the Concentration / Utilities / Binary File Merge menu. Enter hysplit2 as the input name which will be treated as a wildcard to find all the input file names. The output file could be named as the next file in the sequence: hysplit2.006. Because we will be adding (the sum option) four input files (001 -> 005) to create file 006, the multiplier should be changed from 1.0 to 0.20 to convert the sum to an average.

  2. Now press the yellow Create Filenames button and the file INFILE will be created. The base name field will have been erased and the name INFILE will appear in the input name field.The contents of this file should be examined to insure that it contains the desired input files before proceeding. Then press the green Process Files button and the desired output file hysplit2.006 will be created in the working directory.

  3. To automatically compute the statistics of all the ensemble members compared to the measured data, open the Concentration / Display / Ensemble / Statistics menu to open a simple interface requiring only the name of the measured data file and the concentration units conversion factor. After these values are set press the yellow Execute to run the script and display a listing of the output file summarizing the performance statistics of each member, including the ensemble mean (hysplit2.006). A detailed statistical summary for each member is also created in the working directory (sumstat.txt).

When measurement data are available it may be possible to make a more definitive statement regarding the best member. This section describes a more automated way to process and evaluate multiple model simulations. In the case of the meteorological model ensemble shown here, the ensemble mean performs significantly better than any one individual member in four out of the five performance metrics.