Skip to main content
ARS Home » Pacific West Area » Maricopa, Arizona » U.S. Arid Land Agricultural Research Center » Water Management and Conservation Research » Research » Publications at this Location » Publication #269472
Title: Methodology to evaluate the performance of simulation models for alternative compiler and operating system configurations

Author
item Thorp, Kelly
item White, Jeffrey
item PORTER, CHERYL - University Of Florida
item HOOGENBOOM, GERRIT - Washington State University
item NEARING, GREY - University Of Arizona
item French, Andrew

Submitted to: Computers and Electronics in Agriculture
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 12/21/2011
Publication Date: 2/1/2012
Citation: Thorp, K.R., White, J.W., Porter, C.H., Hoogenboom, G., Nearing, G.S., French, A.N. 2012. Methodology to evaluate the performance of simulation models for alternative compiler and operating system configurations. Computers and Electronics in Agriculture. 81(1):62-71.

Interpretive Summary: Applications of computer simulation models used for agricultural and natural resource management increasingly require many thousands of simulations that may take days or weeks to complete. Software testing efforts to increase the computational efficiency and verify the numerical accuracy of such models are important endeavors that will lead to improved robustness, portability, and usefulness of these computational tools. In this study, we assess the performance of the Cropping Systems Model, which is a suite of crop growth simulation models used globally to address complex agricultural problems, by compiling and testing the model with three Fortran compilers on five modern computer operating systems. We demonstrated that certain compiler and operating system combinations offer substantially higher performance than others. We also showed how our software testing methodology can be used to improve programming style and practices, such that the numerical robustness of scientific computer programs can be improved. The methodologies described in the paper are useful for researchers and businesses whose main objective is to provide a robust software product for scientific applications. The results of the study are also specifically useful for researchers, government agencies, and businesses that implement the Cropping Systems Model to address globally relevant agricultural problems, such as climate change, water and nutrient cycling, and risk assessments for crop production.

Technical Abstract: Simulation modelers increasingly require greater flexibility for model implementation on diverse operating systems, and they demand high computational speed for efficient iterative simulations. Additionally, model users may differ in preference for proprietary versus open-source software environments. These issues necessitate the development of strategies to maximize model compatibility across operating systems, to ensure numerically accurate simulations for alternative compiler selections, and to understand how these choices affect computational speed. We developed an approach to evaluate model performance using diverse FORTRAN compilers on multiple computer operating systems. A single desktop computer with five identical hard drives was designed to permit meaningful comparisons between five operating systems while minimizing differences in hardware configuration. Three FORTRAN compilers and relevant software development tools were installed on each operating system. Both proprietary and open-source versions of compilers and operating systems were used. Compatibility and performance issues among compiler and operating system combinations were tested for the Cropping System Model (CSM), as implemented in version 4.5 of the Decision Support System for Agrotechnology Transfer (DSSAT). A simulation study that included 6591 simulations and assessed the full suite of crop growth modules within DSSAT-CSM was conducted for each compiler and operating system configuration. For a given simulation, results were identical for anthesis date (ADAT), maturity date (MDAT), and maximum leaf area index (LAIX) regardless of the compiler or operating system used. Over 97% of the simulations were identical for canopy weight at maturity (CWAM) and cumulative evapotranspiration at maturity (ETCM). Differences in CWAM were predominantly less than 2 kg ha''1 and were likely the result of differences in floating point handling among compilers. Larger CWAM discrepancies highlighted areas for improvement of the model code. Model implementations with the Intel Fortran compiler on the Linux Ubuntu operating system provided the fastest simulations, which averaged 10.9 simulations s-1. Evaluating simulation models for alternative compiler and operating system configurations is invaluable for understanding model performance constraints and for improving model robustness, portability, usefulness, and flexibility.