New record for a large JDG cluster: 500 nodes

Today I ran a 500 node JDG (JBoss Data Grid) cluster on Google Compute Engine, topping the previous record of 256 nodes. The test used is IspnPerfTest [2] and it does the following:

• The data grid has keys [1..20000], values are 1KB byte buffers
• The cache is DIST-SYNC with 2 owners for every key
• Every node invokes 20'000 requests on random keys
• A request is a read (with an 80% chance) or a write (with a 20% chance)
• A read returns a 1KB buffer
• A write takes a 1KB buffer and sends it to the primary node and the backup node
• The test invoker collects the results from all 500 nodes and computes an average which it prints to stdout.

But see for yourself; the YouTube video is here: [1].

In terms of cluster size, I could probably have gone higher: it was pretty effortless to go to 500 instances... any takers ? :-)


[1] http://youtu.be/Cz3CDr31EDU
[2] https://github.com/belaban/IspnPerfTest

Running a JGroups cluster in Google Compute Engine: part 3 (1000 nodes)

A quick update on my GCE perf tests:

I successfully ran a 1000 node UPerf cluster on Google Compute Engine, the YouTube video is at http://youtu.be/Imn1M7EUTGE. The perf tests are started at around 14'13".

I also managed to get 2286 nodes running, however didn't record it... :-)