[Spread-users] Spread Reliability

Jim Vickroy Jim.Vickroy at noaa.gov
Wed Mar 28 10:26:17 EDT 2007

Gordan Bobic wrote:
>>>> You describe reading
>>>> succesfully up to a certain point and then always getting a nothing
>>>> back.  This is quite a bit more specific than "loosing messages" in
>>>> that it seems like you lost the stream.
>>> Indeed, but it still strikes me as a pretty poor design. Dropping a  
>>> client
>>> that can't keep up isn't exactly a "reliable" messaging feature. It is
>>> particularly poor that it does this even with UNRELIABLE_MESS  
>>> messages.
>>> Surely, at least those should be dropped when the queue overflows,  
>>> rather
>>> than disconnecting the client?
>> This is spread's design.  Having a slow reading client slow down all  
>> operations to a crawl isn't usable either.  So, Spread places the  
>> responsibility of flow control on the user.  It has nothing to do  
>> with poor design.  Disconnecting slow clients so that other clients  
>> can continue to operate reliably is intended behaviour, so the design  
>> meets those intentions quite well..
> Not really. In the case where there is only one reader, dropping that 
> reader because the writers are overwhelming it isn't a side-effect I can 
> imagine being useful or desirable.
What is to prevent other subscribers (i.e., "readers") from joining the 
group at any time?  Are you asking for a Spread configuration option to 
essentially provide one-way communication to a single subscriber?
> It also means that if there is a single flooder flooding the network 
> faster than any of the readers can read, e.g. due to a bug in the sending 
> code somewhere, or even just having a sending machine that is so much 
> faster than the others that it is generating messages faster than the 
> listeners can process them, all the listeners get dropped. By the basic 
> rule of concatenation being easier/faster than parsing/splitting, for most 
> processes, readers are likely to be at a disadvantage.
That is why subscribers should only receive/queue messages for 
processing elsewhere (perhaps in another thread).  My view is that it is 
generally less robust to have subscribers performing message receiving 
AND message processing.
>>>> The perl wrapper around the Spread library is really thin XS.  It
>>>> does no interpretive dance :-)  It doesn't much with signals in any
>>>> fashion nor does it intentionally hide return values or side effects
>>>> of the libspread API calls.
>>> Which is all fine, but the fact remains that when you call receive()
>>> without a timeout, any signal traps you set cause the signals to  
>>> disappear completely.
>> Take it up with libspread or perl itself.  The perl wrappers do no  
>> signal handling or sigset modifications.  Assuming that the libspread  
>> SP_receive call doesn't suspend signals, then the issue is with  
>> perl.  FWIW, I have several daemons written in perl that SP_receive  
>> without a timeout and my SIGHUP handlers appear to work just fine.   
>> Perhaps something else is interfering?
> It could be a peculiarity of my perl version (standard FC6), but it only 
> seems to happen with the Spread library.
> Gordan
> _______________________________________________
> Spread-users mailing list
> Spread-users at lists.spread.org
> http://lists.spread.org/mailman/listinfo/spread-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.spread.org/pipermail/spread-users/attachments/20070328/0899d268/attachment.html 

More information about the Spread-users mailing list