<br><div>We have finally got to the bottom of the issue. Fundamentally, a write only client does not exist. If you are connected to spread any client can send a message to any other client directly (non-multicast). This also explains the proffered remedies :</div>
<div> - Adding a process to read messages (Melissa)</div><div> - Increasing the message buffer size (Marcelo) would only make the disconnect less frequent.</div><div><br></div><div>If there is a way to configure a client to ignore all incoming message I would be interested. Otherwise our solution is is line with Melissa's and we have added a process to read any messages.</div>
<div><br></div><div>Many thanks to all for the assistance, it has provided greater insight as well as fixed a nasty bug in our SW.</div><div>##Shawn</div><div><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Wed, Nov 7, 2012 at 3:58 PM, Marcelo San-Martin <span dir="ltr"><<a href="mailto:Marcelo.San-Martin@harmonicinc.com" target="_blank">Marcelo.San-Martin@harmonicinc.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
I used to have a similar problems, in my case I fixed it by increasing MaxSessionMessages in the configuration file. The default value was 1000, I increased it to 10000 and the problem went away.<br>
<br>
Cheers,<br>
Marcelo<br>
<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:spread-users-request@lists.spread.org">spread-users-request@lists.spread.org</a> [mailto:<a href="mailto:spread-users-request@lists.spread.org">spread-users-request@lists.spread.org</a>]<br>
Sent: Wednesday, November 07, 2012 2:02 PM<br>
To: <a href="mailto:spread-users@lists.spread.org">spread-users@lists.spread.org</a><br>
Subject: Spread-users Digest, Vol 91, Issue 4<br>
<br>
Send Spread-users mailing list submissions to<br>
<a href="mailto:spread-users@lists.spread.org">spread-users@lists.spread.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.spread.org/mailman/listinfo/spread-users" target="_blank">http://lists.spread.org/mailman/listinfo/spread-users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:spread-users-request@lists.spread.org">spread-users-request@lists.spread.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:spread-users-owner@lists.spread.org">spread-users-owner@lists.spread.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific than "Re: Contents of Spread-users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: write(): java.net.SocketException (Shawn Bradford)<br>
2. Re: write(): java.net.SocketException (Ed Holyat)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Wed, 7 Nov 2012 11:20:23 -0800<br>
From: Shawn Bradford <<a href="mailto:shawnb@mojix.com">shawnb@mojix.com</a>><br>
<div class="im">Subject: Re: [Spread-users] write(): java.net.SocketException<br>
</div>To: Jonathan Stanton <<a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a>><br>
Cc: <a href="mailto:spread-users@lists.spread.org">spread-users@lists.spread.org</a><br>
Message-ID:<br>
<<a href="mailto:CADTONkdQ4GbQc_nD5oTB4jcpJK5uAbWr7WjRaOrShtdp5W4JVw@mail.gmail.com">CADTONkdQ4GbQc_nD5oTB4jcpJK5uAbWr7WjRaOrShtdp5W4JVw@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<div><div class="h5"><br>
Here is an update on the status of this issue :<br>
- We tried adding 1ms delay between transmissions (still fails)<br>
- We tried upgrading to spread 4.2.0 (still fails)<br>
<br>
We will try Melissa's suggestion and do some reading.<br>
<br>
Thanks,<br>
##Shawn<br>
<br>
<br>
On Mon, Nov 5, 2012 at 3:33 PM, Jonathan Stanton < <a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a>> wrote:<br>
<br>
> Hello Shawn,<br>
><br>
> Since you are using Spread 4.1, this may be a fixed problem. The<br>
> Spread<br>
> 4.2 release that came out in June has a number of fixes (especially to<br>
> the Java API) which solved a number of deadlock, disconnection and crash bugs.<br>
> If you can try the 4.2 release and see if that resolves the problem,<br>
> or look at the changes to the Java API between 4.1 and 4.2 and merge<br>
> them into the version of the Java library that you use that could help.<br>
><br>
> I've included the summary release notes below.<br>
><br>
> Cheers,<br>
><br>
> Jonathan<br>
><br>
> The main new features of this release are:<br>
><br>
> 1) Added Keepalive support to client-server TCP connections. Requires<br>
> correct<br>
> operating system values set for keepalives in order to be useful.<br>
> 2) Switch internal code to use MONOTONIC clocks when available and<br>
> appropriate<br>
> to remove chance of system clock changes (from the clock being set)<br>
> from affecting<br>
> message processing<br>
> 3) Break out events, memory, data_link and alarm code into separate<br>
> libspread-util package. This package also has a number of<br>
> improvements in<br>
> the functionality of those code files which are listed in the internal<br>
> package release notes.<br>
><br>
> It also includes a number of important bug fixes. The most significant<br>
> include:<br>
><br>
> 1) Fix bug with structure size on 64 bit platforms causing crash.<br>
> 2) Fix several deadlock, crashes and race conditions in java Listener code.<br>
> 3) Fix 100 ms timeout in java socket handling code so it does not corrupt<br>
> messages that take a long time to arrive.<br>
> 4) Fix java disconnect bug that prevented client from reconnecting<br>
> until restarted.<br>
> 5) Remove cause of slow message delivery when a client is receiving a<br>
> lot of<br>
> messages and gets into a badger state.<br>
> 6) Improve help output and error messages in utility programs.<br>
> 7) Fix token hurry bug that caused messages to have a 2 second latency in<br>
> specific circumstances.<br>
> 8) Fix crash bug when new daemon configuration files are loaded while the<br>
> system is running.<br>
><br>
><br>
><br>
> -------------------------------------------------------------------------------<br>
> Jonathan Stanton <a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a><br>
> Spread Group Messaging <a href="http://www.spread.org" target="_blank">www.spread.org</a><br>
> Spread Concepts LLC <a href="http://www.spreadconcepts.com" target="_blank">www.spreadconcepts.com</a><br>
><br>
> ----------------------------------------------------------------------<br>
> ---------<br>
><br>
><br>
><br>
> On Nov 5, 2012, at 3:03 PM, Shawn Bradford wrote:<br>
><br>
> > Hello,<br>
> ><br>
> > We are currently using spread and have found this error occurring<br>
> > quite frequently. Unfortunately there is little information on<br>
> > write() errors<br>
> to<br>
> > be found on the net (many more read() errors).<br>
> ><br>
> > *spread.SpreadException: write(): java.net.SocketException:<br>
> > Connection<br>
> reset<br>
> > *<br>
> ><br>
> > Would someone be able to describe what would be a potential issue<br>
> > causing this? I am looking for some guidance as to the source of<br>
> > the error<br>
> (maybe<br>
> > from a developer) to assist in debugging the error.<br>
> ><br>
> > We have tried to write several test apps to replicate the bug but<br>
> > have<br>
> been<br>
> > unsuccessful. Our system is quite large with many moving parts and<br>
> > it is unclear as to what sequence of events are causing the errors.<br>
> ><br>
> > We are using spread 4.1 on 64 bit centos 5.5.<br>
> ><br>
> > Thanks in advance,<br>
> > ##Shawn<br>
> ><br>
> > *--<br>
> > ------------------------------<br>
> > * Director Software | Mojix Inc.<br>
> > phone : <a href="tel:%2B1.562.221.3474" value="+15622213474">+1.562.221.3474</a><br>
> > email : <a href="mailto:shawn.bradford@mojix.com">shawn.bradford@mojix.com</a><br>
> > web : <a href="http://www.mojix.com" target="_blank">www.mojix.com</a><br>
> ><br>
> > Unless expressly identified to the contrary herein, this email and<br>
> > any attachments contain and constitute confidential and<br>
> > proprietary material and information for the sole use of the<br>
> > intended recipient. If you are not the intended recipient or<br>
> > otherwise received this e-mail in error, please (i) immediately<br>
> > delete this email and any attachments,<br>
> print<br>
> > outs and copies of the foregoing and (ii) please notify me<br>
> > immediately by responding to this e-mail message.<br>
> ><br>
> > *<br>
> > _______________________________________________<br>
> > Spread-users mailing list<br>
> > <a href="mailto:Spread-users@lists.spread.org">Spread-users@lists.spread.org</a><br>
> > <a href="http://lists.spread.org/mailman/listinfo/spread-users" target="_blank">http://lists.spread.org/mailman/listinfo/spread-users</a><br>
><br>
><br>
</div></div>-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://lists.spread.org/pipermail/spread-users/attachments/20121107/d3aaafc6/attachment-0001.html" target="_blank">http://lists.spread.org/pipermail/spread-users/attachments/20121107/d3aaafc6/attachment-0001.html</a><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Wed, 7 Nov 2012 16:44:35 -0500<br>
From: Ed Holyat <<a href="mailto:Ed.Holyat@openlink.com">Ed.Holyat@openlink.com</a>><br>
<div class="im">Subject: Re: [Spread-users] write(): java.net.SocketException<br>
</div>To: "<a href="mailto:Shawn.Bradford@mojix.com">Shawn.Bradford@mojix.com</a>" <<a href="mailto:Shawn.Bradford@mojix.com">Shawn.Bradford@mojix.com</a>>, Jonathan<br>
Stanton <<a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a>><br>
Cc: "<a href="mailto:spread-users@lists.spread.org">spread-users@lists.spread.org</a>" <<a href="mailto:spread-users@lists.spread.org">spread-users@lists.spread.org</a>><br>
Message-ID:<br>
<<a href="mailto:648AFB5742D6394FB956DC60697556054491EB0883@OLFANDEXCH01.andover.olf.com">648AFB5742D6394FB956DC60697556054491EB0883@OLFANDEXCH01.andover.olf.com</a>><br>
<br>
Content-Type: text/plain; charset="us-ascii"<br>
<div class="im"><br>
I have not used the java version of spread, but, usually a connection reset means that the connection terminated hard and the other side did not close it. Have you verified that the spread daemon closed the connection on purpose; you can put debugging on the spread daemon to determine if Spread closed the connection because of a slow consumer.<br>
Here are some other scenarios I have scene.<br>
anti virus software delaying the packets and one side getting a Sockettimeoutconnection which wasn't handled correctly, this produced a connection reset on the other side. Try disabling any virus software.<br>
This can also occur if a client terminates before a socket is flushed of all its packets. This can happen on a system with high memory or CPU usage or just sending large packets. Monitor resources and check that the MTU is the same on both sides of the connection.<br>
And there is always the possibility of hardware issues. You can try duplicating the problem outside of Spread by executing ping with a large buffer size ping -t -l 1350 and look for packet loss. This should be performed from client host to daemon host and vise versa<br>
<br>
<br>
From: Shawn Bradford [mailto:<a href="mailto:shawnb@mojix.com">shawnb@mojix.com</a>]<br>
Sent: Wednesday, November 07, 2012 2:20 PM<br>
To: Jonathan Stanton<br>
Cc: <a href="mailto:spread-users@lists.spread.org">spread-users@lists.spread.org</a><br>
Subject: Re: [Spread-users] write(): java.net.SocketException<br>
<br>
<br>
Here is an update on the status of this issue :<br>
- We tried adding 1ms delay between transmissions (still fails)<br>
- We tried upgrading to spread 4.2.0 (still fails)<br>
<br>
We will try Melissa's suggestion and do some reading.<br>
<br>
Thanks,<br>
##Shawn<br>
<br>
</div><div><div class="h5">On Mon, Nov 5, 2012 at 3:33 PM, Jonathan Stanton <<a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a><mailto:<a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a>>> wrote:<br>
Hello Shawn,<br>
<br>
Since you are using Spread 4.1, this may be a fixed problem. The Spread 4.2 release that came out in June has a number of fixes (especially to the Java API) which solved a number of deadlock, disconnection and crash bugs. If you can try the 4.2 release and see if that resolves the problem, or look at the changes to the Java API between 4.1 and 4.2 and merge them into the version of the Java library that you use that could help.<br>
<br>
I've included the summary release notes below.<br>
<br>
Cheers,<br>
<br>
Jonathan<br>
<br>
The main new features of this release are:<br>
<br>
1) Added Keepalive support to client-server TCP connections. Requires correct<br>
operating system values set for keepalives in order to be useful.<br>
2) Switch internal code to use MONOTONIC clocks when available and appropriate<br>
to remove chance of system clock changes (from the clock being set) from affecting<br>
message processing<br>
3) Break out events, memory, data_link and alarm code into separate<br>
libspread-util package. This package also has a number of improvements in<br>
the functionality of those code files which are listed in the internal<br>
package release notes.<br>
<br>
It also includes a number of important bug fixes. The most significant include:<br>
<br>
1) Fix bug with structure size on 64 bit platforms causing crash.<br>
2) Fix several deadlock, crashes and race conditions in java Listener code.<br>
3) Fix 100 ms timeout in java socket handling code so it does not corrupt<br>
messages that take a long time to arrive.<br>
4) Fix java disconnect bug that prevented client from reconnecting until restarted.<br>
5) Remove cause of slow message delivery when a client is receiving a lot of<br>
messages and gets into a badger state.<br>
6) Improve help output and error messages in utility programs.<br>
7) Fix token hurry bug that caused messages to have a 2 second latency in<br>
specific circumstances.<br>
8) Fix crash bug when new daemon configuration files are loaded while the<br>
system is running.<br>
<br>
<br>
-------------------------------------------------------------------------------<br>
</div></div>Jonathan Stanton <a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a><mailto:<a href="mailto:jonathan@spreadconcepts.com">jonathan@spreadconcepts.com</a>><br>
Spread Group Messaging <a href="http://www.spread.org" target="_blank">www.spread.org</a><<a href="http://www.spread.org" target="_blank">http://www.spread.org</a>><br>
Spread Concepts LLC <a href="http://www.spreadconcepts.com" target="_blank">www.spreadconcepts.com</a><<a href="http://www.spreadconcepts.com" target="_blank">http://www.spreadconcepts.com</a>><br>
<div class="im">-------------------------------------------------------------------------------<br>
<br>
<br>
<br>
On Nov 5, 2012, at 3:03 PM, Shawn Bradford wrote:<br>
> Hello,<br>
><br>
> We are currently using spread and have found this error occurring<br>
> quite frequently. Unfortunately there is little information on write()<br>
> errors to be found on the net (many more read() errors).<br>
><br>
> *spread.SpreadException: write(): java.net.SocketException: Connection<br>
> reset<br>
> *<br>
><br>
> Would someone be able to describe what would be a potential issue<br>
> causing this? I am looking for some guidance as to the source of the<br>
> error (maybe from a developer) to assist in debugging the error.<br>
><br>
> We have tried to write several test apps to replicate the bug but have<br>
> been unsuccessful. Our system is quite large with many moving parts<br>
> and it is unclear as to what sequence of events are causing the errors.<br>
><br>
> We are using spread 4.1 on 64 bit centos 5.5.<br>
><br>
> Thanks in advance,<br>
> ##Shawn<br>
><br>
> *--<br>
> ------------------------------<br>
> * Director Software | Mojix Inc.<br>
</div>> phone : <a href="tel:%2B1.562.221.3474" value="+15622213474">+1.562.221.3474</a><tel:%2B1.562.221.3474><br>
> email : <a href="mailto:shawn.bradford@mojix.com">shawn.bradford@mojix.com</a><mailto:<a href="mailto:shawn.bradford@mojix.com">shawn.bradford@mojix.com</a>><br>
> web : <a href="http://www.mojix.com" target="_blank">www.mojix.com</a><<a href="http://www.mojix.com" target="_blank">http://www.mojix.com</a>><br>
<div class="im">><br>
> Unless expressly identified to the contrary herein, this email and any<br>
> attachments contain and constitute confidential and proprietary<br>
> material and information for the sole use of the intended recipient.<br>
> If you are not the intended recipient or otherwise received this<br>
> e-mail in error, please (i) immediately delete this email and any<br>
> attachments, print outs and copies of the foregoing and (ii) please<br>
> notify me immediately by responding to this e-mail message.<br>
><br>
> *<br>
> _______________________________________________<br>
> Spread-users mailing list<br>
</div>> <a href="mailto:Spread-users@lists.spread.org">Spread-users@lists.spread.org</a><mailto:<a href="mailto:Spread-users@lists.spread.org">Spread-users@lists.spread.org</a>><br>
> <a href="http://lists.spread.org/mailman/listinfo/spread-users" target="_blank">http://lists.spread.org/mailman/listinfo/spread-users</a><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://lists.spread.org/pipermail/spread-users/attachments/20121107/f9bc44c7/attachment.html" target="_blank">http://lists.spread.org/pipermail/spread-users/attachments/20121107/f9bc44c7/attachment.html</a><br>
<br>
------------------------------<br>
<div class="im"><br>
_______________________________________________<br>
Spread-users mailing list<br>
<a href="mailto:Spread-users@lists.spread.org">Spread-users@lists.spread.org</a><br>
<a href="http://lists.spread.org/mailman/listinfo/spread-users" target="_blank">http://lists.spread.org/mailman/listinfo/spread-users</a><br>
<br>
<br>
</div>End of Spread-users Digest, Vol 91, Issue 4<br>
*******************************************<br>
<div class="HOEnZb"><div class="h5"><br>
_______________________________________________<br>
Spread-users mailing list<br>
<a href="mailto:Spread-users@lists.spread.org">Spread-users@lists.spread.org</a><br>
<a href="http://lists.spread.org/mailman/listinfo/spread-users" target="_blank">http://lists.spread.org/mailman/listinfo/spread-users</a><br>
</div></div></blockquote></div><br></div>