[Spread-users] Spread scaling

[John Schultz]

I'm pretty sure what is happening here is that your writer application is joining the group to which it is publishing and then never reading.  Spread sees that your application is joined to the group so it delivers the messages back to the application, but the application never calls receive, so the delivered messages never drain out of Spread.  Spread realizes this, sees the queue ever growing and so it disconnects your application when the queue gets to around 1000 messages deep.  The simple solution here is to not have your publishing application join the group to which it publishes.

There is another similar problem you might run into even if your publisher doesn't join the group: if your publisher publishes faster than readers can drain messages from Spread then your subscribers will eventually get disconnected by Spread.  You either need to ensure that you publish "slow enough" that your subscribers can keep up or you need the publisher to join the group, reading messages and conducting a flow control algorithm between itself and all the readers to ensure you don't overwhelm your subscribers.

Cheers!

-----
John Lane Schultz
Spread Concepts LLC
Phn: 301 830 8100
Cell: 443 838 2200

On Mar 9, 2010, at 11:03 AM, Christopher Browne wrote:

Richard Maxwell <rmaxkc at yahoo.com> writes:
> Basically at around 1000 (1k) messages being written to the message
> queue and those messages being read from the queue I get an exception
> that forces the connection closed.  I read on the web someplace that
> spread is compiled with a limit of 1000 messages.  I have not been
> able to find the post I saw with that information.

Are you certain that those messages have actually been taken off the
queue?

When I ran into problems, it was because the queue would only queue 1000
messages, and I didn't have a reader pulling entries off the queue.
(Second order problem: I *did* have a reader, but the writer was
flooding it with more messages than it could cope with.)

You could increase the memory space used for the queue, thereby
increasing the number of messages queueable, but if that merely means
you shift the problem 30 seconds down the road, it's not a particularly
useful solution.
-- 
"cbbrowne","@","ca.afilias.info"
Christopher Browne
"Bother,"  said Pooh,  "Eeyore, ready  two photon  torpedoes  and lock
phasers on the Heffalump, Piglet, meet me in transporter room three"

_______________________________________________
Spread-users mailing list
Spread-users at lists.spread.org
http://lists.spread.org/mailman/listinfo/spread-users