Today's out-of-order processors confront the growing discrepancy of processor and memory speed in part by supporting a large number of in-flight instructions. The hope is to overlap useful computation with long-latency memory accesses. Traditionally, such support is realized largely by increasing the size of pertinent resources such as the register file or the load/store queues. Unfortunately, many such resources cannot scale up arbitrarily without adversely affecting the clock cycle, the pipeline depth, or both. As a result, processors are eventually unable to compensate for the speed gap, and performance suffers. However, a closer look reveals that, behind the apparent need to grow these critical resources to support many in-flight instructions, a significant fraction of such resources are being literally wasted by inactive instructions.

In this talk I will introduce CHeckpointed Early Resource RecYcling (Cherry), a hybrid mode of execution based on ROB and checkpointing that decouples resource recycling and instruction retirement. Resources are recycled early, resulting in a more efficient utilization. Cherry relies on state checkpointing and rollback to service exceptions for instructions whose resources have been recycled. Cherry leverages the ROB to (1) not require in-order execution as a fallback mechanism, (2) allow memory replay traps and branch mispredictions without rolling back to the Cherry checkpoint, and (3) quickly fall back to conventional out-of-order execution without rolling back to the checkpoint or flushing the pipeline.