The Overhead versus Latency Conundrum

When your application performs polling of some kind, you are always confronted with the overhead versus latency conundrum: as a designer, you will be forced to pick the optimal polling period, and hence polling frequency. If responsiveness (low latency) is an issue at all, you will be tempted to reduce the polling interval. Higher polling frequency usually leads to more responsive reaction to the polled stimulus (i.e., lower latency, but at the cost of more CPU overhead [14]).

The amount of CPU overhead may not be obvious however. For the types of applications addressed in this article, the code typically sits in a blocked wait function in between polls. This blocking API can encumber a large amount of overhead for each call. The API may also suffer a minimum time granularity imposed by the operating system, below which timeouts become meaningless. I have seen select calls basically ignore small timeouts, instead blocking for a predetermined minimum when the timeout parameter was below this threshold. If you must poll, it pays to carefully measure the cost of each polling round trip before assuming anything.