[Spread-users] Use MSG_DONTWAIT in sendmsg/sendto options to avoid stuck spread when link is malfunctioning

[Yuval Yeret]

Hi,

We've been using spread for some time in a 2-node Linux environment, where the main link between the two nodes is a 802.3ad trunk driven by Intel iANS driver. 

We've recently upgraded the kernel to Red Hat Enterprise 3 and noticed that when the trunk is down, spread gets blocked at some point in sendmsg() / sendto() to the multicast address. 

We also saw this with our own internal multicast application. 

There is probably some problem with the kernel / Intel drivers ( see RH bugzilla https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=112501, and I'm also working with Intel drivers development on it), but I've found a workaround that works for both my internal application and spread, and which I think should be safe (but am willing to hear other oppinions to the contrary...). 

Attached is a patch which simply adds the option MSG_DONTWAIT to the sendmsg() / sendto() calls in spread. 

      MSG_DONTWAIT
              Enables  non-blocking  operation;  if the operation would block,
              EAGAIN is returned (this can also be enabled  using  the  O_NON-
              BLOCK with the F_SETFL fcntl(2)).


I think the correct behaviour is for the send action to return with error instead of block ...


Happy holidays to everyone, 

Yuval




Copyright (C) 2003 Yuval Yeret

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
    
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR
ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

********************************
<<< file 1: /vobs/vendor/spread/data_link.c@@/main/bug4405/0
>>> file 2: /vobs/vendor/spread/data_link.c
********************************
-----[219 changed to 219]-----
<               ret = sendmsg(chan, &msg, 0);
---
>               ret = sendmsg(chan, &msg, MSG_DONTWAIT);
-----[221 changed to 221]-----
<               ret = sendto(chan, pseudo_scat, total_len, 0,
---
>               ret = sendto(chan, pseudo_scat, total_len, MSG_DONTWAIT,