[Spread-users] producer/consumer queue via spread.

[Allan Bailey]

John Lane Schultz <jschultz at spreadconcepts.com> wrote:
>What I believe is happening is that your producer is producing messages faster 
>than your consumer can take them from Spread.  This causes the consumer's 
>message buffer within Spread to grow.  When it gets too big (~1000 msgs), Spread 
>disconnects your consumer rather than run out of memory and crash.
>
>If you are going to be producing messages fast, then you need a flow control 
>mechanism to ensure you don't overflow your receiver and have Spread kick the 
>receiver.
>
>The simplest flow control mechanism would be to wait a period of time (e.g. - 
>1-10ms) between sending each message.  If your receiver does little work per msg 
>receipt and is not loaded, then it probably will be able to keep up.
>
>For a simple message based flow control mechanism you can examine the 
>spflooder.c program.  For a true flow control mechanism, you need your consumer 
>to send some form of ACKs back to the producer and block the producer from 
>sending if the consumer falls "too far" (i.e. - approaching 1000 msgs) behind.


Ok.  So I think I'm going to have to write a "Queue" protocol that the producer
puts a limited set of items in the queue, and waits for consumers to ack them.

The next problem is this:  how can I guarantee that only 1 consumer consumes 1 item?

I guessing that I will need to work out the ACK aspect of the
protocol.  I was hoping there would be someway to have multiple
consumers take an item and ACK it w/o having to check for the race
condition.  Any pointers would be appreciated.

thanks,
Allan



>Cheers!
>
>Allan Bailey wrote:
>> John Lane Schultz <jschultz at spreadconcepts.com> wrote:
>>> That solution only works if no one else is sending to the group and you have 
>>> group membership turned off.
>>>
>>> If you really want just a producer, with no reads, then the most simple and 
>>> efficient route is to simply not SP_join to the group.
>> 
>> Yes, this works for the producer, but now the consumer also dies after a bit.
>> 
>> I've tried having the consumer also send messages, but it still dies.
>> 
>> thanks,
>> Allan
>> 
>
>-- 
>John Schultz
>Spread Concepts LLC
>Phn: 443 838 2200
>Fax: 301 560 8875
>
>_______________________________________________
>Spread-users mailing list
>Spread-users at lists.spread.org
>http://lists.spread.org/mailman/listinfo/spread-users
>
--
.allan.