Belas Blog: Buy one, get many for free: message batching in JGroups

Just a quick heads-up of what's going on in JGroups 3.3 with message batching.

Currently, when the transport receives a message bundle (say 20 messages), it passes the bundle to the regular thread pool (OOB messages are never bundled). The thread which handles the bundle grabs each message, adds it to the retransmission table, and then removes as many messages as possible and passes them up one-by-one.

So, although we have a message bundle, we don't process it as a bundle, but rather add each message individually and then pass them up up one-by-one.

Message batching [1] changes this. It reads a message bundle directly into 2 MessageBatch instances: one for OOB messages and one for regular messages. (This already shows that OOB messages are now bundled, too, but more on this later). The OOB MessageBatch is passed to the OOB thread pool for handling, the regular batch to the regular pool.

A message batch is nothing more than a list of messages.

A message batch is handled by only 1 thread: the thread passes the entire batch up the stack. Each protocol can remove messages it consumes (e.g. FD_ALL), change messages in-place (e.g. COMPRESS or ENCRYPT), remove and add messages (e.g. FRAG2), remove messages and pass up a new batch (NAKACK2, UNICAST2) or even do nothing (SIZE, STATS).

The advantage is that a protocol can now handle many messages at once, amortizing (e.g.) lock acquisition costs. For example, NAKACK2 adds all 20 messages to the retransmission table at once, thereby acquiring the table lock only once rather than 20 times. This means that we incur the cost of 1 lock acquition, instead of 20. It goes without saying that this will also reduce lock contention, at least in this particular case, even if the lock duration will be slightly longer than before.

I'll present some performance numbers soon, but so far preliminary performance tests look promising !

So while message bundling queues messages and sends them across the wire as a list, but stops at the receiver's transport; message batching takes this idea further and passes that bundle up all the way to the channel. (Note that this will require another receive() callback in the Receiver, but this will be transparent by default).

Message batching will allow other cool things to happen, e.g.

OOB messages will be bundled too now. If no bundling is desired, tag a message as DONT_BUNDLE.
We can simplify the message bundler (on the sender side), see [2]. As a result, I might even be able to remove all existing 4 message bundlers. As you know, I like removing stuff, and making code easier to read !
RPC responses can be bundled [3]
UNICAST2 can now ack the 'last message' [4]