[Spread-users] Receiver disconnection and app-level flow control

[Tudor Dumitras]

Hello everybody,

 From reading some of the older messages on this list (especially 
http://lists.spread.org/pipermail/spread-users/2002-March/000655.html) I 
learned that if a process is too slow in receiving and processing Spread 
messages, it will be disconnected from the Spread daemon. My 
understanding is that, because Spread implements reliable communication 
semantics, it will never drop messages. If a sender is faster than a 
receiver, then the receiver-side Spread daemon will buffer some of the 
messages and, when this buffer fills up (the length of the buffer is 
defined by MAX_SESSION_MESSAGES in spread_params.h), the daemon will 
close the connection of the slow process. Therefore, the application is 
responsible for implementing some form of flow control to prevent the 
messages from being sent faster than they can be received.

It seems to me that the easiest (for the application developer) method 
of flow control would be the the TCP-like behavior of blocking the 
sender if the receiver is too slow / not ready. In this scenario, 
instead of disconnecting the process that calls SP_receive() too seldom, 
the SP_multicast() call would block if the destination group contains a 
member that cannot receive.

I realize that Spread does not support this behavior and I would like to 
understand the reason. More specifically, I have 6 questions:

- Would blocking the sender instead of disconnecting the receiver break 
the extended virtual synchrony semantics?

- If not, do the architecture or the implementation of Spread make this 
approach infeasible?

- Is it possible to poll the sender's daemon to find out if it is "safe" 
to send (i.e., if this won't cause any receiver to be kicked out)?

- When does the sender buffer (who's length is defined by the WATER_MARK 
parameter from spread_params.h) fill up, causing the sender to block?  
More precisely, how can this happen independently of the receivers' 
ability to receive fast enough?

- How can Spread be modified to implement the blocking sender behavior?

- What other method of flow control would you recommend?

I would appreciate any kind of comments that you might have on these 
issues. Spread seems to be a very powerful and well implemented tool, 
but I am not sure how to use it to get the best results. I am gradually 
learning this and your help will be greatly appreciated.


Tudor

-- 
______________________________________

Tudor A. Dumitras

ECE Department
Carnegie Mellon University
(412) 268-5005

http://www.ece.cmu.edu/~tdumitra