Methodological Notes on the Regional Level Validation of a Microscopic Traffic Simulation Model

Traffic simulation models have been increasingly used to evaluate and compare alternative complex real-world traffic problems. Simulation is safer, less expensive and faster than field testing. The past few years have witnessed substantial development of transportation network modeling tools and stronger emphasis on addressing the need to model large-scale networks more accurately and efficiently. While these simulation models can be helpful to transportation engineers, the models must be well calibrated and validated before they can provide credible results. However, simulation models have been often conducted under default parameters. This is mainly due to either the difficulties in field data collection or the lack of knowledge of the appropriate procedure to calibrate and validate traffic simulation models. This paper presents the results of a recent effort to microscopically simulate the regional evacuation plan for New Orleans Metropolitan Area during the hurricane Katrina. The model involved over 300,000 vehicles moving within a road network that covered several thousand square miles over a 48 hour period. Output statistics were generated on a second-by-second basis for each traveler in the system. Model validation was based upon a comparison of the TRANSIMS generated traffic volumes to the corresponding traffic volumes actually observed during the 2005 hurricane Katrina evacuation. The validation process included the percent error estimation and the regression analysis between the simulated and observed traffic volume data. This study was unique in that it is among the first to develop validation criteria for a regional model based on actual traffic data collected during a live regional mass evacuation. Analysis was performed utilizing percent errors estimation based on direct comparisons of the hourly volumes at each counting station. Also, an alternative validation approach was carried out using regression analysis between the cumulative observed and simulated volumes for the same stations by analyzing the fit for the regression line y =a + bx + ε. The error percentage and the fit were found to be reasonable with an error percentage less than 25 percent and an R-squared value of over 0.80. This indicated that the TRANSIMS simulation model was a realistic representation of the evacuation operations observed during the hurricane Katrina.