For years, the steadily growing clock speed of new processors has been relied upon to deliver increased performance for a wide range of applications consistently. However, processor frequency scaling has stalled, driving microprocessor companies to investigate adding value through alternative technologies. To continue making gains in system performance existing systems need to optimize execution dynamically.
Dynamic code optimizers are a type of runtime systems that modify an application at run-time to promote desirable execution characteristics, such as high performance, low power, or better-managed resource consumption on the target platform. Most dynamic optimizers first observe run-time features to determine frequently executed sequences of code for performing optimizations within a given instance of execution. These sequences can span procedure boundaries as well as across shared libraries and therefore have more potential for optimization. After accumulating sufficient knowledge, the optimizers regenerate/translate the code into better versions to achieve the desired program/system behavior.
The typical architecture of a runtime system is shown above. We build systems that monitor execution using hardware and software counters, and then dynamically optimize the code by invoking optimizers that specialize in code transformations. Typically, only the hot code is optimized as the penalty paid in optimizing the hot code is quickly recouped or amortized over the improvements we observe from the optimized code execution. We apply these techniques in the context of binary translators, just-in-time compilers for managed languages like JS and Java to improve performance, power, and reliability.