[Spread-users] Spread Reliability
Jim Vickroy
Jim.Vickroy at noaa.gov
Wed Mar 28 10:26:17 EDT 2007
Gordan Bobic wrote:
>>>> You describe reading
>>>> succesfully up to a certain point and then always getting a nothing
>>>> back. This is quite a bit more specific than "loosing messages" in
>>>> that it seems like you lost the stream.
>>>>
>>> Indeed, but it still strikes me as a pretty poor design. Dropping a
>>> client
>>> that can't keep up isn't exactly a "reliable" messaging feature. It is
>>> particularly poor that it does this even with UNRELIABLE_MESS
>>> messages.
>>> Surely, at least those should be dropped when the queue overflows,
>>> rather
>>> than disconnecting the client?
>>>
>> This is spread's design. Having a slow reading client slow down all
>> operations to a crawl isn't usable either. So, Spread places the
>> responsibility of flow control on the user. It has nothing to do
>> with poor design. Disconnecting slow clients so that other clients
>> can continue to operate reliably is intended behaviour, so the design
>> meets those intentions quite well..
>>
>
> Not really. In the case where there is only one reader, dropping that
> reader because the writers are overwhelming it isn't a side-effect I can
> imagine being useful or desirable.
>
What is to prevent other subscribers (i.e., "readers") from joining the
group at any time? Are you asking for a Spread configuration option to
essentially provide one-way communication to a single subscriber?
>
> It also means that if there is a single flooder flooding the network
> faster than any of the readers can read, e.g. due to a bug in the sending
> code somewhere, or even just having a sending machine that is so much
> faster than the others that it is generating messages faster than the
> listeners can process them, all the listeners get dropped. By the basic
> rule of concatenation being easier/faster than parsing/splitting, for most
> processes, readers are likely to be at a disadvantage.
>
>
That is why subscribers should only receive/queue messages for
processing elsewhere (perhaps in another thread). My view is that it is
generally less robust to have subscribers performing message receiving
AND message processing.
>>>> The perl wrapper around the Spread library is really thin XS. It
>>>> does no interpretive dance :-) It doesn't much with signals in any
>>>> fashion nor does it intentionally hide return values or side effects
>>>> of the libspread API calls.
>>>>
>>> Which is all fine, but the fact remains that when you call receive()
>>> without a timeout, any signal traps you set cause the signals to
>>> disappear completely.
>>>
>> Take it up with libspread or perl itself. The perl wrappers do no
>> signal handling or sigset modifications. Assuming that the libspread
>> SP_receive call doesn't suspend signals, then the issue is with
>> perl. FWIW, I have several daemons written in perl that SP_receive
>> without a timeout and my SIGHUP handlers appear to work just fine.
>> Perhaps something else is interfering?
>>
>
> It could be a peculiarity of my perl version (standard FC6), but it only
> seems to happen with the Spread library.
>
> Gordan
>
>
> _______________________________________________
> Spread-users mailing list
> Spread-users at lists.spread.org
> http://lists.spread.org/mailman/listinfo/spread-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.spread.org/pipermail/spread-users/attachments/20070328/0899d268/attachment.html
More information about the Spread-users
mailing list