[Spread-users] problem with spread/mod_log_spread/spreadlogd

[Theo Schlossnagle]

John Schultz wrote:

> So, in effect, you would end up doing some form of crude end-to-end 
> flow control but rather than proactively preventing problems you are 
> reactively responding to problems.  Plus, it might be difficult to 
> effectively ramp senders back up once you've throttled them.
>
> The real answer if you think your system can have consistent flows 
> that will overwhelm receivers is end-to-end flow control.  I discussed 
> in detail a few such approaches that can help mitigate this problem 
> and I sketched some ways to truly solve the problem (shared global 
> window):

In the particular case of system logging, the solution is a big more 
simple.  The concept here would be to separate the logging process from 
the app and the publishing process to Spread and communicate over a 
durable local message store.  So, mod_log_spread would be mod_log_jlog 
which would write to journalled "write-ahead-logs" and then the Spread 
publishing would be a standalone jlog2spread daemon that is responsible 
for orchestrating flow control to receivers.  You would need to define 
how the semantics of your local durability translate into delivery 
guarantees on the Spread side, but that's just a choice of semantics.

We do something along these lines in our clustering product.  It works 
well, handles the not-infrequent Spread ring collapse and also bursty 
behaviour that might overwhelm a receiver and, of course, out-right 
reciever failures.

In general people don't feel that attached to their web logs, but I 
suppose they should.  mod_log_jlog would be a good replacement for 
mod_log_spread eventually.

-- 
// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Ecelerity: Run with it. -- http://www.omniti.com/