[Spread-users] 1 problem with Spread.pm and 1 problem with spread daemon, was

[George Schlossnagle]

On Friday, January 18, 2002, at 12:02 AM, Tom Mornini wrote:

> On Thursday, January 17, 2002, at 07:19 PM, George Schlossnagle wrote:
>
>>> 1) Spread.pm Perl module returns a Perl undef from multicast() in 
>>> some circumstances. I've seen it when I call multicast() repeatedly 
>>> without calling receive() in circumstances when there are incoming 
>>> errors messages. It *might* be related to issue #2 below, however.
>>
>> probably is.
>
> Turns out, it's not.
>
> Here's a super simple test case:
>
> create a process that just sends continually, and kill off the spread 
> server. The multicast() subroutine returns Perl undef. I'm also nearly 
> 100% certain that this happens when calling multicast() repeatedly when 
> the receive buffer overflows and the Spread daemon disconnects you 
> (case #2 below).

Are you joined to any rings?  Do you have membership messages coming to 
you?  Can you send your test case?

>
>>> 2) Somewhat more mysteriously, receive() only processes get 
>>> spontaneously disconnected and receive() returns a CONNECTION_CLOSED 
>>> correctly whenever a receiver is hammered continuously from more than 
>>> 1 process on the same box. The sending processes DO NOT get 
>>> disconnected, however.
>>
>> Each spread daemon maintains a separate queue for each client 
>> connected to it.  If the client's queue exceeds the max size, it (that 
>> specific client) will be disconnected.  You can lengthen the queue at 
>> compile time, but you probably just need to make sure you process it 
>> faster.
>
> OK...
>
> But doesn't that seem like the wrong way to handle something like this? 
> All those queued messages are LOST!
>
> Why not just block all senders until there's some headroom (a percent 
> of the max queue available?) and blast something into the logs to show 
> that the system is throttling?
>
> Or, return an error message that says the system is throttled...
>
> Both of these are obviously non-trivial as all the spread daemons need 
> to be synchronized on the throttling, but that's what Spread is all 
> about, isn't it? :-)
>
> On Thursday, January 17, 2002, at 08:52 PM, Guido van Rossum wrote:
>
>>> The Spread daemon has a hard limit of 1000 queued messages.  If you're
>>> behind reading your messages, and Spread finds that it's queued up
>>> more than 1000, it disconnects you.  I just found this out myself, and
>>> it is causing us a major rewrite of our app. :-(
>
> How can you rewrite the app to fix this? It seems like the only way to 
> fix it is to: 1) Spread less, or 2) empty mailboxes faster...


fwiw, you can alter this in the spread header files.  I have mine jacked 
up to 20000.  But I know what my memory situation is like and how big my 
messages are.

>
> On Thursday, January 17, 2002, at 04:27 AM, Jeremy Hylton wrote:
>
>> We've just been discussing the same problem with our Spread app.  I'm
>> not sure what the right limit is, but there is a problem with making
>> it too big, right?  As I understand it, the daemon has to buffer all
>> those messages in memory until the client takes them.
>
> Well, at least we're not alone! :-)
>
> --
> -- Tom Mornini
> -- eWingz Systems, Inc.
>
>
>
> _______________________________________________
> Spread-users mailing list
> Spread-users at lists.spread.org
> http://lists.spread.org/mailman/listinfo/spread-users
>