Artificial intelligence and machine learning are experiencing widespread adoption in the industry, academia, and even public consciousness. This has been driven by the rapid advances in the applications and accuracy of AI through increasingly complex algorithms and models; this, in turn, has spurred research into developing specialized hardware AI accelerators. The rapid pace of the advances makes it easy to miss the forest for the trees: they are often developed and evaluated in a vacuum without considering the full application environment in which they must eventually operate. In this paper, we deploy and characterize Face Recognition, an AI-centric edge video analytics application built using open source and widely adopted infrastructure and ML tools. We evaluate its holistic, end-to-end behavior in a production-size edge data center and reveal the “AI tax” for all the processing that is involved. Even though the application is built around state-of-the-art AI and ML algorithms, it relies heavily on pre- and post-processing code which must be executed on a general-purpose CPU. As AI-centric applications start to reap the acceleration promised by so many accelerators, we find they impose stresses on the underlying software infrastructure and the data center’s capabilities: storage and network bandwidth become major bottlenecks with increasing AI acceleration. By not having to serve a wide variety of applications, we show that a purpose-built edge data center can be designed to accommodate the stresses of accelerated AI at 15% lower TCO than one de-rived from homogeneous servers and infrastructure. We also discuss how our conclusions generalize beyond Face Recognition as many AI-centric applications at the edge rely upon the same underlying software and hardware infrastructure.
The Edge Computing Lab resides in the John A. Paulson School of Engineering and Applied Sciences at Harvard University under the direct supervision of Prof. Vijay Janapa Reddi.
Edge computing is a computing paradigm where computation is performed mostly on power-constrained devices with limited processing capability, or on-premise datacenters. Edge systems only loosely depend on large-scale cloud computing resources as needed (i.e., via computational offloading). In the edge computing context, our interests span from the low-power Internet of Things (IoT) devices to mobile computing devices like smartphones, all the way up to high-performance edge servers for various application domains like autonomous cars and robotics.
We are a team of computer system architects who specialize in edge computing platforms. Our work spans both hardware and software to solve fundamental computing challenges in the context of edge computing and its various endpoints. We are specifically good at understanding the interactions across the circuits, architecture and software layers. Our philosophy is to be driven by the problems we want to solve, rather than by the medium (hardware or software) through which we can solve the problems. Hence, we are only limited by our ability to creatively address the problems and not by our apriori knowledge of a particular problem domain.