Throughput vs Latency

Is throughput a trade off against latency, or can I improve both?

In this talk we look at a common trade off made between Throughput and Latency, processing a single message at a time, or processing a batch of messages.

First terminology is covered so that we understand what Latency really is and the other factors at play. Then we look at the standard patterns adopted to make the tradeoffs.

With this in mind we take a deeper look at the parts of a system we can measure and hopefully control, including message read and write time & processing time. For example, what impact does the serialization format you choose, and how you use it, impact your performance? Does arrival rate and distribution into your queues affect your response time? How does my language of choice affect my performance profile?

If you plan to just throw "cloud scale" at the problem, where do you fall off the cliff of scale-ability? If you add more processing power, how wide can you partition your datastore? What is the performance penalty of coordination, and when does that cost more than the cost of processing a message?

To conclude with, we see that Latency and Throughput are not tradeoffs you need to make. In fact there are for more important decisions that need your focus.