[Spread-users] consequences of making a large group

[John Lane Schultz]

Paul Rubel wrote:
> I'm considering adding some functionality to my system that would
> entail adding every spread-using application in my system into a
> single group. This group would be used to occasionally disseminate
> name-service like information.

This is actually a fairly common practice.

> Does the size of a group
> really matter or is the number of daemons and the network topology the
> real limiter? Thinking about the ring topology it seems the total size
> could be more important than the size of any group.

In Spread the more limiting factor tends to be the number of daemons and the 
network topology.

Spread can support groups of up to about 2000 members out of the box.  The 
limiting factor is that membership view changes have to fit into a single Spread 
message and (in the worst case) each member (32 bytes) can be listed twice.  If 
you increase the size of the max message, then you (I believe) increase the 
limit on the number of group members as well.

The main effect of having groups with very many members is that the group 
membership list for each group gets large (obviously).  Therefore, if your 
clients track membership, then the Spread daemon has to push large messages (e.g 
- 32-64KB for 1000 members) to each of the members every time there is a 
membership.  Furthermore, often the more clients you have in a group, the more 
often group memberships occur.  If you have a lot of clients connecting to the 
same daemon, then you can begin to see quadratic performance degradation in 
membership changes because the group list grows linearly w/ the size of the 
group and if you have (a fraction of) linear clients connecting to a single 
daemon that causes it to generate linear number of membership messages; so the 
effects multiply to become quadratic.

> Does the group structure have any effect on recovery time when a host
> is lost? I'm guessing that it doesn't as the underlying structure
> needs to be repaired regardless of any group membership.

The only effect would be on in memory data structure operations would should be 
very fast (i.e. - not noticeable) relative to the network protocol.  Of course, 
the load from the operations discussed above (delivering membership messages to 
clients) could affect performance noticeably.

> Currently we have 9 hosts with 10s of groups. Would 30 hosts 100
> groups change the answer?

It shouldn't.  The more important questions are how many clients are going to be 
in your system, what is the maximum number of clients connecting through a 
single daemon and how many clients are going to sit in that one mega group?

Good luck!

-- 
John Schultz
Spread Concepts LLC
Phn: 443 838 2200
Fax: 301 560 8875