[Spread-users] Re: Question about unexpected spread behaviour
Uma Chingunde
umac at jhu.edu
Wed Aug 22 18:00:50 EDT 2007
The email seems to have been blocked maybe because of the log files so
resending the log files as text.
---------- Forwarded message ----------
From: Uma Chingunde <umac at jhu.edu>
Date: Aug 22, 2007 5:47 PM
Subject: Question about unexpected spread behaviour
To: Spread-users at lists.spread.org
Hi,
I have a Spread configuration between 2 hosts and I am seeing some weird
behavior between them.
Since the network does not support broadcast I have configured both hosts as
separate sites.
My spread.conf looks like this:
--------
Spread_Segment x.x.x.255:4899 {
uma-vm-1 x.x.x.105
}
Spread_Segment x.x.x.255:4899 {
uma-vm-2 x.x.x.106
}
---------------------
I have a test application that communicates using spread.
However at certain intervals the second daemon seems to partition away and
then re-merge when no network change has occurred.
I initially thought that the application was sending a wrong message to
spread that was causing the problem. However it doesn't seem to be the case.
The first daemon's log file (spread1.log) shows both daemons as always being
in the same partition.
The log files for the partitioned daemon (spread2.log) shows the segments
occasionally in different partitions.
The relevant snippets are below and the log files are attached.
Does anyone have an idea about why I am seeing such behavior?
I can't figure out why one daemon would see a network partition differently
from the other, if such a partition was occurring which I am pretty sure in
this case is not.
Is there a configuration issue that I am missing somewhere?
Any help would be appreciated.
Thanks,
Uma
Log file snippet for first daemon
------------------------------------------
Conf_load_conf_file: My name: uma-vm-1, id: x.x.x.105, port: 4899
Membership id is ( 168918633, 1187811520)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811620)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811682)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811800)
--------------------
Log file snippet for second daemon
---------------------------------------
Membership id is ( 168918633, 1187811520)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811620)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918634, 1187811665)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
0 x.x.x.255 4899
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811682)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.spread.org/pipermail/spread-users/attachments/20070822/3541f9c8/attachment.html
-------------- next part --------------
/===========================================================================\
| The Spread Toolkit. |
| Copyright (c) 1993-2006 Spread Concepts LLC |
| All rights reserved. |
| |
| The Spread toolkit is licensed under the Spread Open-Source License. |
| You may only use this software in compliance with the License. |
| A copy of the license can be found at http://www.spread.org/license |
| |
| This product uses software developed by Spread Concepts LLC for use |
| in the Spread toolkit. For more information about Spread, |
| see http://www.spread.org |
| |
| This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF |
| ANY KIND, either express or implied. |
| |
| Creators: |
| Yair Amir yairamir at cs.jhu.edu |
| Michal Miskin-Amir michal at spreadconcepts.com |
| Jonathan Stanton jstanton at gwu.edu |
| John Schultz jschultz at spreadconcepts.com |
| |
| Major Contributors: |
| Ryan Caudy rcaudy at gmail.com - contribution to process groups.|
| Claudiu Danilov claudiu at acm.org - scalable, wide-area support. |
| Cristina Nita-Rotaru crisn at cs.purdue.edu - GC security. |
| Theo Schlossnagle jesus at omniti.com - Perl, autoconf, old skiplist. |
| Dan Schoenblum dansch at cnds.jhu.edu - Java interface. |
| |
| Special thanks to the following for discussions and ideas: |
| Ken Birman, Danny Dolev, Jacob Green, Mike Goodrich, Ben Laurie, |
| David Shaw, Gene Tsudik, Robbert VanRenesse. |
| |
| Partial funding provided by the Defense Advanced Research Project Agency |
| (DARPA) and the National Security Agency (NSA) 2000-2004. The Spread |
| toolkit is not necessarily endorsed by DARPA or the NSA. |
| |
| For a full list of contributors, see Readme.txt in the distribution. |
| |
| WWW: www.spread.org www.spreadconcepts.com |
| Contact: info at spreadconcepts.com |
| |
| Version 4.00.00 Built 29/November/2006 |
\===========================================================================/
Conf_load_conf_file: using file: /usr/local/etc/spread.conf
Successfully configured Segment 0 [x.x.x.255:4899] with 1 procs:
uma-vm-1: x.x.x.105
Successfully configured Segment 1 [x.x.x.255:4899] with 1 procs:
uma-vm-2: x.x.x.106
ENABLING Dangerous Monitor Commands! Make sure Spread network is secured
Finished configuration file.
Hash value for this configuration is: 628536559
Conf_load_conf_file: My name: uma-vm-1, id: x.x.x.105, port: 4899
Membership id is ( 168918633, 1187811520)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811620)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811682)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811800)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811950)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812023)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812052)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812174)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812273)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812379)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812582)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812788)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812817)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187813001)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187813279)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187813482)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187813683)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187813878)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187814081)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187814109)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187814287)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187814492)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187814694)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187814897)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187815099)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187815301)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187815506)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187815709)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187815909)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187816110)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187816308)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187816510)
--------------------
Configuration at uma-vm-1 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership i
-------------- next part --------------
/===========================================================================\
| The Spread Toolkit. |
| Copyright (c) 1993-2006 Spread Concepts LLC |
| All rights reserved. |
| |
| The Spread toolkit is licensed under the Spread Open-Source License. |
| You may only use this software in compliance with the License. |
| A copy of the license can be found at http://www.spread.org/license |
| |
| This product uses software developed by Spread Concepts LLC for use |
| in the Spread toolkit. For more information about Spread, |
| see http://www.spread.org |
| |
| This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF |
| ANY KIND, either express or implied. |
| |
| Creators: |
| Yair Amir yairamir at cs.jhu.edu |
| Michal Miskin-Amir michal at spreadconcepts.com |
| Jonathan Stanton jstanton at gwu.edu |
| John Schultz jschultz at spreadconcepts.com |
| |
| Major Contributors: |
| Ryan Caudy rcaudy at gmail.com - contribution to process groups.|
| Claudiu Danilov claudiu at acm.org - scalable, wide-area support. |
| Cristina Nita-Rotaru crisn at cs.purdue.edu - GC security. |
| Theo Schlossnagle jesus at omniti.com - Perl, autoconf, old skiplist. |
| Dan Schoenblum dansch at cnds.jhu.edu - Java interface. |
| |
| Special thanks to the following for discussions and ideas: |
| Ken Birman, Danny Dolev, Jacob Green, Mike Goodrich, Ben Laurie, |
| David Shaw, Gene Tsudik, Robbert VanRenesse. |
| |
| Partial funding provided by the Defense Advanced Research Project Agency |
| (DARPA) and the National Security Agency (NSA) 2000-2004. The Spread |
| toolkit is not necessarily endorsed by DARPA or the NSA. |
| |
| For a full list of contributors, see Readme.txt in the distribution. |
| |
| WWW: www.spread.org www.spreadconcepts.com |
| Contact: info at spreadconcepts.com |
| |
| Version 4.00.00 Built 29/November/2006 |
\===========================================================================/
Conf_load_conf_file: using file: /usr/local/etc/spread.conf
Successfully configured Segment 0 [x.x.x.255:4899] with 1 procs:
uma-vm-1: x.x.x.105
Successfully configured Segment 1 [x.x.x.255:4899] with 1 procs:
uma-vm-2: x.x.x.106
ENABLING Dangerous Monitor Commands! Make sure Spread network is secured
Finished configuration file.
Hash value for this configuration is: 628536559
Conf_load_conf_file: My name: uma-vm-2, id: x.x.x.106, port: 4899
Membership id is ( 168918633, 1187811520)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811620)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918634, 1187811665)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
0 x.x.x.255 4899
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811682)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811800)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187811950)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918634, 1187812027)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
0 x.x.x.255 4899
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812023)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918634, 1187812080)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
0 x.x.x.255 4899
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812052)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812174)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918634, 1187812275)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
0 x.x.x.255 4899
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812273)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812379)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812582)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1187812788)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
1 x.x.x.255 4899
uma-vm-1 x.x.x.105
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918634, 1187812785)
--------------------
Configuration at uma-vm-2 is:
Num Segments 2
0 x.x.x.255 4899
1 x.x.x.255 4899
uma-vm-2 x.x.x.106
====================
Membership id is ( 168918633, 1
More information about the Spread-users
mailing list