AI Tax in Mobile SoCs: End-to-end Performance Analysis of Machine Learning in Smartphones

Citation:

M. Buch, Z. Azad, A. Joshi, and V. J. Reddi, “AI Tax in Mobile SoCs: End-to-end Performance Analysis of Machine Learning in Smartphones,” in 2021 IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS '21, Virtual, Stony Brook, NY, March 28-30, 2021.
PDF1.25 MB

Abstract:

Mobile software is becoming increasingly feature rich, commonly being accessorized with the powerful decision making capabilities of machine learning (ML). To keep up with the consequently higher power and performance demands, system and hardware architects add specialized hardware units onto their system-on-chips (SoCs) coupled with frameworks to delegate compute optimally. While these SoC innovations are rapidly improving ML model performance and power efficiency, auxiliary data processing and supporting infrastructure to enable ML model execution can substantially alter the performance profile of a system. This work posits the existence of an AI tax, the time spent on non-model execution tasks. We characterize the execution pipeline of open source ML benchmarks and Android applications in terms of AI tax and discuss where performance bottlenecks may unexpectedly arise.

IEEE Version