[Spread-users] Problem with segement in spread

[Akezyt Janedittakarn]

Hi,

Thank you very much Ryan. I still have some more questions or responses to
your questions. They are inlined below. Comments and advices from you would
help. 

Regards,

Ake

-----Original Message-----
From: Ryan Caudy [mailto:caudy at jhu.edu] 
Sent: Sunday, November 16, 2003 3:02 AM
To: Akezyt Janedittakarn
Cc: spread-users at lists.spread.org
Subject: Re: [Spread-users] Problem with segement in spread

Hi,

Responses inlined below.

Cheers,
Ryan

> 
> 1) The first question is "does segmentation in spread provides message 
> (traffic) isolation between segments?" For example, if there are two 
> segments, Seg1 and Seg2, and there is one group, G1. If a node on Seg1 
> sends a message to G1 and there are no members of G1 in Seg2, should I 
> see that message at the nodes in Seg2? From spread's documentation and 
> papers, that should not be the case. However, in my following settings, 
> I still see that traffic on Seg2. The configuration file is.
> 

By nodes, I assume you mean Spread daemons.  Yes, you should see the 
message reach those daemons, even though they don't have clients who 
need to receive them.  This is how Spread makes sure that all daemons 
have the information they need to correctly meet ordering requirements, 
although the daemons who won't need to deliver the message to clients 
really only need the metadata contained in the header.  Pulling off the 
body of the message is done as an optimization in the wide-area research 
version of Spread, but not in the open source version.

== More responses/questions

Since what I am doing is measuring the amount of traffic cause by different
style of server replications, I would like to measure only the traffic that
is deem necessary. The problem that I'm facing right now is when servers
talk to each others by sending message to 'servers' group (all of them are
on Seg1), the daemon at the client machine (on Seg2) also receive those
messages (and discarding them). This is fine at the application level, but
that make a lot of different at the network level (because I measure the
traffic). What I really want is to confine those messages to segment that
has server daemon running. For me it seems like it is possible to do that
(from spread's paper). Is that possible in the open source version of spread
(or the research version of spread)? 
> 
> Spread_Segment 192.168.1.255:5122{
> 
> S1 192.168.1.105
> 
> S2 192.168.1.106
> 
> }
> 
> Spread_Segment 192.168.1.127:5122{
> 
> S4 192.168.1.201
> 
> }
> 

To make sure that I understand your configuration, these machines are in 
two separate /25 subnets, with mask 255.255.255.128 and broadcast 
addresses as listed in the segment names?

== More responses/questions

No they are on the same subset (and on the same Ethernet segment).


> 
> The steps that I took are as follows:
> 
> - I ran spread daemon on only S1 and S2.
> 
> - Next I ran a server program on S1 which creates/joins a 'servers' group.
> 
> - I ran a client program on S2 sending messages to 'servers' group.
> 
> - No spread daemons are running on S4
> 
> - Then I used 'ethereal' to capture the traffic at S4. It got around 16 
> frames per S2's request (all of them are broadcast messages). This is
fine.
> 

What does the traffic look like?  I would expect to see the occasional 
unicast probe message from S1 to S4 (attempting to find a daemon at S4 
and build a membership that includes it), but thats it (until you add a 
daemon to S4).


> - Next I ran spread daemon on S4. I captured traffic at S4 again, this 
> time I got I got around 400 frames per S2 request. Most of the frames 
> are unicast message directly sent to S4 from S2 and unicast message from 
> S4 to S1. It seemed like they formed a ring which is contradict to its 
> documentation because it is across segment.
>  

Do you mean 400 ethernet frames?  This seems large... what size are the 
messages sent by the client at S2?  How long were your tests, and how 
often was S2 sending?

They definitely formed a ring.  In this version of Spread, all daemons 
are organized into a ring to pass the token, with order defined by the 
order in the configuration file (so it stays within a segment until it 
finishes, and then is passed to the next).  So, there will be unicasts 
of the token from S1 to S2, from S2 to S4, and from S4 to S1.  Also, 
message traffic between segments is unicast, so you should see 
additional traffic between the segments from S2 to S4.

== More responses/questions

Sorry for confusion, I meant 400 udp packets. The size of the message is
20KB and the sending rate is 1 message every 5 seconds. Also let me rephrase
my settings. S1 and S2 are in the same segment while S4 is in a separate
segment.
 - When you said "all daemons", do you mean all daemons in all segments will
form a single global ring? My current understanding from reading spread's
paper is that only daemons in the same segment will form a ring for that
segment and the entire system is a tree of segments. However, the above
experiment showed that S4 is also part of the same ring as S1 and S2 which
are on different segments. Furthermore, the "spread wide area group
communication systems" paper page 7 states that "Pruning: packets are not
forwarded to child links of the tree which have no members interested in the
packet". That is why I am so confusing about the result of my setting.

> 
> 2) Is it necessary to have 2 segments in 2 separate subnets?
> 
> Currently, both segments are on the same Ethernet segment (connected to 
> the same hub). I am not sure that this should matter or not.
> 

Its not necessary.  On a network like you mentioned, it would make more 
sense to have one segment, but if the subnet organization must remain 
the way you described it in your configuration file, you'll have to use 
IP multicast instead of broadcast in order to do so.

== More responses/questions

The reason that I create two segments because I believed that spread will
create one ring for each segment.

> 
> 3) Is there any flag or configuration option that I can use to make 
> spread suppress those unnecessary messages?
> 

The messages aren't unecessary, as explained (at a high level) above. 
The best answer I can give you is not to run daemons where you know that 
you won't have clients.

== More responses/questions

I have to run daemon on S4 because I have to send message from that node.
However, I don't want messages destined to 'servers' group to go to this
daemon.

> 
> I have been stuck on this problem for a while now, and would really 
> appreciate any answers and advice.
> 

Hope this helps you.

-- 
Ryan W. Caudy
Center for Networking and Distributed Systems
Department of Computer Science
Johns Hopkins University