[Spread-users] problem with spread/mod_log_spread/spreadlogd

[John Schultz]

Hey Scott,

We've gotten requests for such a feature in the past.  Unfortunately, it 
seems that there really isn't any good way around clients doing their own 
end-to-end flow control as appropriate for their applications.

The problem with your suggestion is that you are attempting to detect and 
respond to a flow control problem on the receiving end rather than the 
sending end.  In the end, the only way to truly solve a flow control 
problem is to slow ALL of the *SENDERS* down to a flow that (all) the 
receivers can withstand.  One application might decide that slowing down 
to the slowest receiver is a good idea.  Another application might want to 
maintain a certain throughput and if some machines can't keep up then "too 
bad, so sad" and Spread should kick them.

Regardless, let's say we took your suggestion.  What would a receiver do 
when it receives such a warning message?  It could certainly slow down its 
own sending, but through what mechanism and how much would it throttle 
itself?  To affect other senders that might not be having buffer problems 
it has to send a message out saying "Hey, slow down!"

So, in effect, you would end up doing some form of crude end-to-end flow 
control but rather than proactively preventing problems you are reactively 
responding to problems.  Plus, it might be difficult to effectively ramp 
senders back up once you've throttled them.

The real answer if you think your system can have consistent flows that 
will overwhelm receivers is end-to-end flow control.  I discussed in 
detail a few such approaches that can help mitigate this problem and I 
sketched some ways to truly solve the problem (shared global window):

http://lists.spread.org/pipermail/spread-users/2002-March/000655.html

Look at the thread of messages with email subject [1225].

---
John Schultz
Spread Concepts
Phn: 443 838 2200

On Wed, 7 Sep 2005, Scott Barvick wrote:

> John,
>
> Has anyone given any thought to passing daemon or connection congestion
> status back through through messages received by the clients?  I can
> easily kill my Java client with too much traffic, but I might be able to
> slow down the traffic if I knew I was starting to get into to trouble.
> Having a client close with an error is more disruptive in some cases
> that cutting back on the data sent so it would be nice to have that
> option.
>
> Thanks,
> Scott