Speaker: Mark Heinrich
Affiliation: ECE, Cornell University
Date: 2/1/01
Time and Location: 4:15pm, B11 Kimball Hall
Title: Simulation vs. Reality: The Importance of Building Hardware

Simulation is the primary method for evaluating computer systems during all
phases of the design process. One significant problem with simulation is that it rarely models the system exactly, and quantifying the resulting simulator error can be difficult. More importantly, architects often assume without proof that although their simulator may make inaccurate absolute performance predictions, it will still accurately predict architectural trends.

In this talk we present the source and magnitude of error in a range of architectural simulators by comparing the simulated execution time of several applications and microbenchmarks to their execution time on the actual hardware being modeled. The existence of a hardware gold standard allows us to find, quantify, and fix simulator inaccuracies. We then use the simulators to predict architectural trends and analyze the sensitivity of the results to the simulator configuration. We find that most of our simulators predict trends accurately, as long as they model all of the important performance effects for the application in question. Unfortunately, it is difficult to know what these effects are without having
a hardware reference, as they can be quite subtle. This calls into question the value, for architectural studies, of highly detailed simulators whose characteristics are not carefully validated against a real hardware design.

During the final portion of the talk, I will discuss my current research efforts in the Computer Systems Laboratory, including a vision of the future for high-performance multiprocessors.