New record for a large JGroups cluster: 1538 nodes

The largest JGroups cluster so far has been 536 nodes [1], but since last week we have a new record: 1538 nodes !

The specs for this cluster are:

• 4 X Intel Xeon CPU E3-1220 V2 @ 3.10GHz 4GB: 8 boxes of 13 members = 104
• 1 X Intel Celeron M CPU 440 @ 1.86GHz 1GB: 8 boxes of 3 members = 24
• 1 X Intel Celeron M CPU 440 @ 1.86GHz 2GB: 141 boxes of 10 members =1410
• Total = 1538 members
• JGroups 3.4.0.Beta2 (custom build with JGRP-1705 [2] included)
• 120 MB of heap for each node

The time to form a 1538 node cluster was around 115 seconds and it took a total 11 views (due to view bundling being enabled).

Note that the physical memory listed for the hardware above is shared by all the nodes on that box, e.g. in the last HW config of 141 boxes, 10 nodes share the 2GB of physical memory; each member has 120 MB max of heap available.

The picture below shows the management GUI. The last line shows member number 1538, which took roughly 11 seconds to join (due to view bundling) and the total time for the cluster to form was ca. 115 seconds.

[1] http://belaban.blogspot.ch/2011/04/largest-jgroups-cluster-ever-536-nodes.html
[2] https://issues.jboss.org/browse/JGRP-1705