Specialised processors.
WHAT ACCELERATORS ARE
Processors designed for specific classes of computation.
WHAT THE COMMON ONES ARE
Graphics processors, repurposed for general computation Purpose-built machine learning accelerators Field-programmable devices, configured for a task
WHAT THEY PROVIDE
Far greater throughput on suitable workloads.
WHAT SUITABLE MEANS
Many identical operations on large data Regular access patterns Limited branching
WHAT THEY DO NOT SUIT
Irregular, branching, sequential computation.
WHAT THE TRANSFER COST IS
Moving data between main memory and the accelerator, which is frequently the bottleneck.
WHAT THAT MEANS
Transferring data to perform a small computation is slower than not using the accelerator.
WHAT TO DO ABOUT IT
Keep data resident and perform many operations on it.
WHAT PORTING CODE REQUIRES
Restructuring for parallelism, not merely recompiling.
WHAT LIBRARIES PROVIDE
Accelerated implementations of common operations, requiring no low-level work.
WHAT TO TRY FIRST
Those.
WHY
Most of the benefit, with a fraction of the effort.
WHAT TO MEASURE
End-to-end time, including transfers.
