The summary.
SPEEDUP IS LIMITED BY THE PART THAT CANNOT BE PARALLELISED
A program ninety per cent parallel cannot exceed a tenfold speedup however many processors are added. Reduce the serial portion first.
MEMORY ACCESS DOMINATES MOST SCIENTIFIC CODE, NOT ARITHMETIC
Processors are far faster than memory, so structuring data for the access pattern matters more than clever computation.
Use established numerical libraries — they are tuned per hardware and extremely difficult to match.
IN DISTRIBUTED SYSTEMS YOU FREQUENTLY CANNOT TELL SUCCESS FROM FAILURE FROM SLOWNESS
The response is the same: nothing. That is why timeouts, retries and operations safe to repeat are unavoidable.
Never order events by timestamp across machines.
DATA MOVEMENT IS INCREASINGLY THE CONSTRAINT, NOT COMPUTATION
Compute where the data is. And with accelerators, transfer cost frequently exceeds the benefit unless data stays resident.
CLOUD IS CHEAPER AT LOW UTILISATION AND MORE EXPENSIVE AT HIGH
Establish expected utilisation honestly before choosing, and set budget alerts from the start.
ESTIMATE JOB RESOURCES ACCURATELY
Overstated requirements queue longer; understated ones are terminated partway.
