[Spread-users] 1 problem with Spread.pm and 1 problem with spread daemon, was

[Tom Mornini]

On Thursday, January 17, 2002, at 07:19 PM, George Schlossnagle wrote:

>> 1) Spread.pm Perl module returns a Perl undef from multicast() in some 
>> circumstances. I've seen it when I call multicast() repeatedly without 
>> calling receive() in circumstances when there are incoming errors 
>> messages. It *might* be related to issue #2 below, however.
>
> probably is.

Turns out, it's not.

Here's a super simple test case:

create a process that just sends continually, and kill off the spread 
server. The multicast() subroutine returns Perl undef. I'm also nearly 
100% certain that this happens when calling multicast() repeatedly when 
the receive buffer overflows and the Spread daemon disconnects you (case 
#2 below).

>> 2) Somewhat more mysteriously, receive() only processes get 
>> spontaneously disconnected and receive() returns a CONNECTION_CLOSED 
>> correctly whenever a receiver is hammered continuously from more than 
>> 1 process on the same box. The sending processes DO NOT get 
>> disconnected, however.
>
> Each spread daemon maintains a separate queue for each client connected 
> to it.  If the client's queue exceeds the max size, it (that specific 
> client) will be disconnected.  You can lengthen the queue at compile 
> time, but you probably just need to make sure you process it faster.

OK...

But doesn't that seem like the wrong way to handle something like this? 
All those queued messages are LOST!

Why not just block all senders until there's some headroom (a percent of 
the max queue available?) and blast something into the logs to show that 
the system is throttling?

Or, return an error message that says the system is throttled...

Both of these are obviously non-trivial as all the spread daemons need 
to be synchronized on the throttling, but that's what Spread is all 
about, isn't it? :-)

On Thursday, January 17, 2002, at 08:52 PM, Guido van Rossum wrote:

>> The Spread daemon has a hard limit of 1000 queued messages.  If you're
>> behind reading your messages, and Spread finds that it's queued up
>> more than 1000, it disconnects you.  I just found this out myself, and
>> it is causing us a major rewrite of our app. :-(

How can you rewrite the app to fix this? It seems like the only way to 
fix it is to: 1) Spread less, or 2) empty mailboxes faster...

On Thursday, January 17, 2002, at 04:27 AM, Jeremy Hylton wrote:

> We've just been discussing the same problem with our Spread app.  I'm
> not sure what the right limit is, but there is a problem with making
> it too big, right?  As I understand it, the daemon has to buffer all
> those messages in memory until the client takes them.

Well, at least we're not alone! :-)

--
-- Tom Mornini
-- eWingz Systems, Inc.