[Spread-users] fatal error

[Nolan Johnson]

We've been occasionally having issues that cause all of the spread daemons in our network (8 daemons) to go down more or less simultaneously.    Unfortunately, we've not been able to get logs about any of them, but shortly (20 minutes or so) after all went down and were restarted, 1 of the 8 went down, with logs as follows.  My assumption is that this is the same problem that caused all to go down, or at least related.

Log:
  [Thu 20 Mar 2008 20:39:57] Prot_handle_token: BUG WORKAROUND: Too many rounds in EVS state; swallowing token; state:
  [Thu 20 Mar 2008 20:39:57]      Aru:              85
  [Thu 20 Mar 2008 20:39:57]      My_aru:           85
  [Thu 20 Mar 2008 20:39:57]      Highest_seq:      102
  [Thu 20 Mar 2008 20:39:57]      Highest_fifo_seq: 15
  [Thu 20 Mar 2008 20:39:57]      Last_discarded:   85
  [Thu 20 Mar 2008 20:39:57]      Last_delivered:   85
  [Thu 20 Mar 2008 20:39:57]      Last_seq:         3435
  [Thu 20 Mar 2008 20:39:57]      Token_rounds:     501
  [Thu 20 Mar 2008 20:39:57] Last Token:
  [Thu 20 Mar 2008 20:39:57]      type:             0x80040080
  [Thu 20 Mar 2008 20:39:57]      transmiter_id:    180619528
  [Thu 20 Mar 2008 20:39:57]      seq:              0
  [Thu 20 Mar 2008 20:39:57]      proc_id:          180619528
  [Thu 20 Mar 2008 20:39:57]      aru:              85
  [Thu 20 Mar 2008 20:39:57]      aru_last_id:      180619526
  [Thu 20 Mar 2008 20:39:57]      flow_control:     2
  [Thu 20 Mar 2008 20:39:57]      rtr_len:          20
  [Thu 20 Mar 2008 20:39:57]      conf_hash:        -822413227
  [Thu 20 Mar 2008 20:40:04] Discard_packets: (EVS before transitional) packet 86 not exist Exit caused by Alarm(EXIT)
  


Our general setup:
All 8 nodes live in a single multicast segment:

Spread_Segment  228.2.3.4 {
    node1    10.x.x.1
    node2    10.x.x.2
    node3   10.x.x.3
    node4    10.x.x.4  
    node5    10.x.x.5
    node6    10.x.x.6
    node7    10.x.x.7
    node8    10.x.x.8
}
DaemonUser = nobody
DaemonGroup = nogroup
DebugFlags = { PRINT EXIT }
EventPriority = INFO
EventLogFile = /wherever/spread.log
EventTimeStamp

No other configs in config file.  Running on Ubuntu 7.04, Spread 4.0.0.


Any idea what's going on, or any hints for debugging this further?

Thanks.

Nolan Johnson

       
---------------------------------
Looking for last minute shopping deals?  Find them fast with Yahoo! Search.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.spread.org/pipermail/spread-users/attachments/20080320/3c1fad51/attachment.html