[Spread-users] problem with spread/mod_log_spread/spreadlogd
jschultz at commedia.cnds.jhu.edu
Wed Sep 7 17:32:42 EDT 2005
We've gotten requests for such a feature in the past. Unfortunately, it
seems that there really isn't any good way around clients doing their own
end-to-end flow control as appropriate for their applications.
The problem with your suggestion is that you are attempting to detect and
respond to a flow control problem on the receiving end rather than the
sending end. In the end, the only way to truly solve a flow control
problem is to slow ALL of the *SENDERS* down to a flow that (all) the
receivers can withstand. One application might decide that slowing down
to the slowest receiver is a good idea. Another application might want to
maintain a certain throughput and if some machines can't keep up then "too
bad, so sad" and Spread should kick them.
Regardless, let's say we took your suggestion. What would a receiver do
when it receives such a warning message? It could certainly slow down its
own sending, but through what mechanism and how much would it throttle
itself? To affect other senders that might not be having buffer problems
it has to send a message out saying "Hey, slow down!"
So, in effect, you would end up doing some form of crude end-to-end flow
control but rather than proactively preventing problems you are reactively
responding to problems. Plus, it might be difficult to effectively ramp
senders back up once you've throttled them.
The real answer if you think your system can have consistent flows that
will overwhelm receivers is end-to-end flow control. I discussed in
detail a few such approaches that can help mitigate this problem and I
sketched some ways to truly solve the problem (shared global window):
Look at the thread of messages with email subject .
Phn: 443 838 2200
On Wed, 7 Sep 2005, Scott Barvick wrote:
> Has anyone given any thought to passing daemon or connection congestion
> status back through through messages received by the clients? I can
> easily kill my Java client with too much traffic, but I might be able to
> slow down the traffic if I knew I was starting to get into to trouble.
> Having a client close with an error is more disruptive in some cases
> that cutting back on the data sent so it would be nice to have that
More information about the Spread-users