[Spread-users] unicast performance on multiple spread daemon

[Jonathan Stanton]

On Tue, Aug 27, 2002 at 12:44:40AM +0900, ÀÌÁ¤¼ö wrote:
> I use spread for unicast communication as well as group communication. 
> And the most important thing in the our system is the performance.
>  
> I think that running a daemon on each machine is better solution than single daemon configuration
> for better performace because of  following considerations. (Of coure it is better solution for system robustness, 
> but I interested in the performance in this article.)

Yes, it should provide better performance.

>  
> i) the system load will be distributed to multiple daemon.
> ii) each application can connect to daemon by IPC(using "4803" as daemon name)
> iii) if both sender and receiver run on the same machine, there is no inter-daemon communication
>  
> But, all expectaction is not true. specially, iii) is not true.

No, iii cannot be true with Spread, even though in isolation of unicast
local IPC it may appear possible. As you state below, Spread treats
unicast as a special case of multicast and as such it needs to get the
token before sending. 

Once it has the token, however, it can send a number of messages back to
back with no additional communication. 
> 
> Which case shows better speed? The answer is case 1.

This will always be true, at least minimally.  By running a second daemon
you require the daemons to coordinate at least some as you 'CAN' send
group multicasts (and it sounds like you do send some, you just also send
unicasts). 
> 
> The cause of this result may be overhead by token message exchange(Ring Protocol).
> token message is exchanged between daemon  WHENEVER unicast send is occur.
> In the group communication, it is reasonable. 
> But, in the unicast communication using private group name, each daemon know the member of private group
> and the location of the member without exchanging token message. Therefore, daemon maybe need not 
> token message or reduce it.

It is true the daemon can know this. We did not put a special case in the
code to detect and try to optimize this scenerio because it is not a
common case for group communication. 

What you are actually seeing below is node .236 requesting the token and
then receiving it (which are unicast messages) and then sending the actual
data message onto the network with a multicast. 

It would not be too hard to add a check to the code to not send the actual
multicast if the message is a unicast to a local private group, so you
could try that and see how much it improves the performance. 
> 
> Even if both of sender and receiver is runnig on machine1(local communication), token message
> is exchanged between daemon on every send opreation. snoop result (at machine1) is below.
> Is it unnecessary, ins'n it?
> 
> -=-=-=-
> 192.168.1.236 -> ASM2_ge0     UDP D=4804 S=56223 LEN=32
>     ASM2_ge0 -> 192.168.1.236 UDP D=4804 S=32936 LEN=32
>         ASM2 -> 225.0.1.2    UDP D=4803 S=32936 LEN=136
> -=-=-
> In this case, one multicast UDP, two broadcast UDP is occur for every send operation.

It should be 2 unicast udp for each "set" of messages and one multicast
for each message. If you are sending unicast's quickly for each token you
get you should be able to send upto 15 messages with the default flow
control. You can raise that number in the monitor.

> my opinion is right? or is any fault in my configuration, test method?

Mostly right. As I said above the local machine unicast is an unusual case
for a group communciation system. We did optimize unicast fairly well, and
if you are sending fast the token cost should be minimal compared with the
actual multicasts. 

Jonathan

-- 
-------------------------------------------------------
Jonathan R. Stanton         jonathan at cs.jhu.edu
Dept. of Computer Science   
Johns Hopkins University    
-------------------------------------------------------