Parallel slowdown

A diagram of the program runtime (shown in blue) and program speed-up (shown in red) of a real-world program with sub-optimal parallelization. The dashed lines indicate optimal parallelization–linear increase in speedup and linear decrease in program runtime. Note that eventually the runtime actually increases with more processors (and the speed-up likewise decreases). This is parallel slowdown.

Parallel slowdown is a phenomenon in parallel computing where parallelization of a parallel computer program beyond a certain point causes the program to run slower (take more time to run to completion).

Parallel slowdown is typically the result of a communications bottleneck. As more processing nodes are added, each processing node spends progressively more time doing communication than useful processing. At some point, the communications overhead created by adding another processing node surpasses the increased processing power that node provides, and parallel slowdown occurs.

This computer science article is a stub. You can help Wikipedia by expanding it.