[Spread-users] performance again

Yair Amir yairamir at cnds.jhu.edu
Tue Aug 6 17:33:48 EDT 2002


David> Following on:
David> I am rapidly approaching the situation where our Spread managed cluster 
David> will be expanded to 60 nodes with
David> Spread being used, ideally, for both reliability and messaging.

David> Messaging will be order (100 x 6k messages)  / second, comprising, 
David> typically, pairs of (broadcast to worker nodes, one unicast to master node)
David> I intend for each node to be running a spread daemon.

David> I imagine there will be a scalability problem with Spread at this number 
David> of nodes:


>>3.16.2 has no problem to support 100 receivers but not if each of them
>>is running on a different machine. For that you need a smarter design.


David> Can somebody summarise the issues affecting scalability ,and what of the
David> following could increase the chances, if any, of Spread supporting the 
David> above configuration

For 3.16.2

David> 1- no of nodes running spread daemon
I am not sure it will work well with 60 just the way it is, but it might.

David> 2- message rate , bandwidth
In your case, these are really not a limitation - they are fairly far
from what the system should be able to achieve.

David> 3- reducing the message 'reliability' type
This will not change anything other than some latency.

David> 4- moving the unicast to a standard socket call
I am not sure if that would help much. It is more complicated in terms
of settings and I am not sure how much it would help performance,
especially in that rate.

David> 5- moving the multicast to say, LGMP,LGCP,RMT.. i.e. having Spread just 
David> maintain the reliability
As before, message rate and bandwidth are not a limitation in your
case. Even if they were, I don't see how the above protocols could
help you with Spread. You can build a different message bus
altogether.

David> 6- A smarter design, or a more scalable version.
A smarter design is a way to set up the system so that what you need
works. If you really have a problem already, you might consider contacting
Spread Concepts.

David> What are the issues with 'A smarter design, or a more scalable version'?
David> Is this design
David> - planned,
David> - implemented,
David> - available from the commercial arm of Spread.

In your case, as you seem to be interested only in a clustered
environment, and as your message rate does not seem to be too high
I would think the current version can address it with some smarter
design of the network.

David> If available, is it a panacaea or an improvement

Spread Concepts has a version that is scalable with a focus on wide
area networks. It can be a good solution also for clustered networks
that reach their limitation with the current daemon.
However, it seems to me you can do without it based on what you stated.

David> so many questions......

David> Thanks in anticipation,

David> David Turland

Cheers,

       :) Yair.
       









David> _______________________________________________
David> Spread-users mailing list
David> Spread-users at lists.spread.org
David> http://lists.spread.org/mailman/listinfo/spread-users





More information about the Spread-users mailing list