Why graphics processors are different.
WHAT A PROCESSOR OPTIMISES FOR
Low latency on sequential, branching work.
WHAT A GRAPHICS PROCESSOR OPTIMISES FOR
Throughput on many identical operations.
HOW IT ACHIEVES THAT
Very many simple cores Execution in groups, all performing the same instruction High memory bandwidth Latency hidden by switching between many pending groups
WHAT DIVERGENCE IS
Threads within a group taking different branches.
WHAT IT COSTS
Both paths executed, with results discarded for the inactive threads.
WHY THAT MATTERS
Heavily branching code performs badly, sometimes worse than on a processor.
WHAT MEMORY COALESCING IS
Threads in a group accessing adjacent memory, combined into few transactions.
WHAT SCATTERED ACCESS COSTS
Many separate transactions, and a large fraction of achievable bandwidth lost.
WHAT SHARED MEMORY PROVIDES
Fast memory shared within a group, under program control.
WHAT IT SUITS
Data reused by several threads.
WHAT DETERMINES PERFORMANCE MOST
Memory access patterns, not arithmetic.
WHAT THE TRANSFER COST IS
Moving data between main memory and device memory.
WHAT TO DO ABOUT IT
Keep data resident, and perform many operations on it.
