Conferences and supporting programme
Benchmarking the Intelligent Edge – A Framework to Measure Embedded AI Performance
'Intelligent Edge' is Artificial Intelligence (AI) processing located at the data source, as opposed to running at a remote computing server (cloud computing). The performance requirements for 'Intelligent Edge' are stringent. In order to compare the performance of such systems, the Institute of Embedded Systems (InES) at Zurich University of Applied Sciences (ZHAW) developed a framework for benchmarking AI hardware. The benchmark consists of tests evaluating the performance of individual machine learning operation, commonly used patterns and complete well known models. This allows to better compare hardware for AI and make a sounder decision when choosing a hardware for deployment in production. The benchmark was tested on different hardware platforms utilizing GPUs or dedicated AI-accelerators and can easily be adapted to other platforms, since the benchmarks are based on elemental tensorflow operations. It features ways to use custom compilers and data loaders, to facilitate testing AI-accelerators which require a non-standard workflow. As proof of concept for the benchmark, multiple hardware platforms were evaluated, showing the strengths and weaknesses of devices. These devices encompass among others, a Jetson TX2, a Raspberry Pi and a Movidius neural compute stick. Paper and presentation describe the challenges of AI benchmarking, the approaches and the framework of the benchmarking. Benchmark results run on existing systems are presented and discussed.
--- Date: 27.02.2019 Time: 16:00 - 16:30 Location: Conference Counter NCC Ost