How to hijack a JGroups channel inside Infinispan / JBoss ... (and get away with it !)

Have you ever used an Infinispan or JBoss (WildFly) cluster ?
If no --> skip this article.

Have you ever had the need to send messages around the cluster, without resorting to RpcManager offered by Infinispan or HAPartition provided by WildFly ?
If no --> skip this article.

Have you ever wanted to hijack the JGroups channel used by Infinispan and WildFly and use it for your own purposes ?
If no --> skip this article.

I recently added light-weight channels [1] to JGroups. They provide the ability to send and receive messages over an existing channel, yet those messages are private to the light-weight channel. And the hijacked channel doesn't see those private messages either.

This is good when an application wants to reuse an existing channel, and doesn't want to create/configure/maintain its own channel, which would increase resource consumption.

Also, applications can create many (hundreds) of light-weight channels, as they don't use a lot of resources and are easy and quick to create and destroy.

Even better: if an application would like to add capabilities to an existing stack, e.g. atomic counters, a distributed execution service or distributed locking, then it can define the protocol(s) it wants to add and the private channel will add these. Again, the hijacked channel is unaffected by this.

There's documentation on this at [1], so let me show you how to hijack a channel inside of an Infinispan application. The application can be downloaded from [3].

The (original but edited) code that creates an Infinispan cache looks like this:

    protected void start() throws Exception {
        mgr=new DefaultCacheManager("infinispan.xml");
        cache=mgr.getCache("clusteredCache");
        eventLoop();
    }

It creates a CacheManager, then creates a Cache instance off of it. This is it. Now, we want to grab the JGroups channel and hijack it to run a Draw instance on it (HijackTest:53, [4]):


    protected void start() throws Exception {
        mgr=new DefaultCacheManager("infinispan.xml");
        cache=mgr.getCache("clusteredCache");
        Transport   tp;
        ForkChannel fork_ch;
        tp=cache.getAdvancedCache().getRpcManager().getTransport();
        Channel main_ch=((JGroupsTransport)transport).getChannel();
        fork_ch=new ForkChannel(main_ch, 
                                "hijack-stack", 
                                "lead-hijacker",
                                true,
                                ProtocolStack.ABOVE,
                                FRAG2.class);
        Draw draw=new Draw((JChannel)fork_ch);
        fork_ch.connect("ignored");
        draw.go();
        eventLoop();
    }

The code in bold was inserted. So what does it do ? It grabs the JGroups channel from the Infinispan cache, creates a light-weight (fork) channel  and runs Draw [5] over it. Draw is a replicated whiteboard GUI application. To explain this step-by-step:

• First the Transport instance is retrieved from the cache, then the JGroups channel from it
• Next a ForkChannel is created. It is passed the hijacked JGroups channel, the name of the newly created fork stack ("hijack-stack") and the name of the fork channel ("lead-hijacker"). Then we state that we want to dynamically insert a FORK protocol if it doesn't exist, above FRAG2. FORK [2] is needed to create fork channels
• Next, we create an instance of Draw (shipped with JGroups) on the newly created fork channel
• Then we connect the fork channel and call go() on Draw which starts up the GUI

The result is shown below:

The hijacked application is a text-based Infinispan perf test (shown in the 2 shell windows). The hijacking Draw windows are shown above. Pressing and moving the mouse around will send multicasts of coordinates/color around the cluster over the fork channel, but the hijacked channel will not notice anything.

Seem like we got away with hijacking the JGroups channel after all ! :-)



[1] http://www.jgroups.org/manual/html/user-advanced.html#ForkChannel

[2] https://issues.jboss.org/browse/JGRP-1613

[3] https://github.com/belaban/HijackTest

[4] https://github.com/belaban/HijackTest/blob/master/src/main/java/org/dubious/HijackTest.java#L53

[5] https://github.com/belaban/JGroups/blob/master/src/org/jgroups/demos/Draw.java