<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7226.0">
<TITLE>Re: [Spread-users] Sess_validate_read_header: Message has illegal type field 0x80000080</TITLE>
</HEAD>
<BODY>
<DIV id=idOWAReplyText38203 dir=ltr>
<DIV dir=ltr><FONT face=Arial color=#000000 size=2>Jonathon,</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2></FONT> </DIV>
<DIV dir=ltr><FONT face=Arial size=2>You are correct that I was talking about
the C code this time. The C code does push the TCP socket code up to the
system maximum, but as you note it is still possible to overflow that.
When you say that the overflow condition is handled correctly, do you mean in
the way that we observe: Sess_read gets invalid header data and then
closes the connection? Do you think there is anything we could do on
the SP_multicast side to detect that the length would overflow the TCP buffer
and then possibly wait? Are there ioctls calls that could do this before a
send?</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2></FONT> </DIV>
<DIV dir=ltr><FONT face=Arial size=2>Thanks,</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2>Scott</FONT></DIV></DIV>
<DIV dir=ltr><BR>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Jonathan Stanton
[mailto:jonathan@cnds.jhu.edu]<BR><B>Sent:</B> Thu 4/21/2005 5:48
PM<BR><B>To:</B> Scott Barvick<BR><B>Cc:</B>
spread-users@lists.spread.org<BR><B>Subject:</B> Re: [Spread-users]
Sess_validate_read_header: Message has illegal type field
0x80000080<BR></FONT><BR></DIV>
<DIV>
<P><FONT size=2>Mentioning this reminded me of something. I remember in the past
there<BR>was a problem someone had that went away if they increased the
tcp<BR>socket_buffers on the client-server tcp connections. If you are
using<BR>Java, the code that increases the socket buffers is commented out in
the<BR>library we distribute becuase it wasn't compatible with old JVM's,
but<BR>if you are running anything current you can uncomment it and
increase<BR>the buffer sizes.<BR><BR>Scott, I thought you were using the C
library, not java? Is that<BR>correct? The C library does increase the socket
buffer size, although it<BR>can still be filled, and that should be handled
correctly.<BR><BR>Jonathan<BR><BR>On Thu, Apr 21, 2005 at 05:32:41PM -0400,
Scott Barvick wrote:<BR>> After more experimentation and upgrading to 3.17.3
(which still showed<BR>> the problem), I believe this is the same as the Java
problems - we are<BR>> trying to send more data into the TCP socket than it
can hold. With big<BR>> sends and high rates, a temporary increase in
load on the daemon will<BR>> cause the clients to overrun the TCP
buffer. This resulted in the<BR>> Sess_read reading in all 0s for the
header and then appropriately<BR>> calling it invalid.<BR>><BR>> Is
there any way for the SP_multicast() to monitor the TCP queue and<BR>> block
on a send until there is enough room?<BR>><BR>> Thanks,<BR>>
Scott<BR>><BR>> On Tue, 2005-04-19 at 20:18, Ryan Caudy wrote:<BR>>
> The interaction there seems fairly normal -- if I had to guess, I<BR>>
> would say that this points to a memory-corrupting bug. I assume
this<BR>> > is version 3.17.2 or 3.17.3?<BR>> ><BR>> >
Cheers,<BR>> > Ryan<BR>> ><BR>> > On 4/19/05, Scott Barvick
<sbarvick@revasystems.com> wrote:<BR>> > > Greetings,<BR>>
> ><BR>> > > I'm getting the following error when running with a
few test systems,<BR>> > > and I'm curious if others have seen anything
similar. I believe we are<BR>> > > hitting it medium hard with
sends between 2 systems. When I disconnect<BR>> > > one system
with a hard stop of the app, the other system sees this,<BR>> > >
processes the membership changes, but then a short time later kills the<BR>>
> > client session when it receives a type field that has no type bits
set<BR>> > > (only the endian bit - 0x80000080).<BR>> >
><BR>> > > I turned on SESSION and GROUP debug logging and included
the output<BR>> > > below. I was looking through the code to see
how a message can get<BR>> > > through without the (FIFO_MESS |
SELF_DISCARD) bits set as we send them<BR>> > > with the SP_multicast()
call. It probably is significant that the group<BR>> > > just
dropped from 2 members to 1 member (the sender), but this works<BR>> >
> fine in the steady state operation, even with only one member.<BR>> >
><BR>> > > Any similar experience or thoughts?<BR>> >
><BR>> > > Thanks,<BR>> > > Scott<BR>> > ><BR>>
> > -------------------<BR>> > ><BR>> > > [...] lots
more where this came from<BR>> > ><BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
***** Other system goes down ******<BR>> > > Send_join: State is
4<BR>> > > Send_join: State is 4<BR>> > > Memb_handle_token:
handling form2 token<BR>> > > Handle_form2 in FORM<BR>> > >
Memb_transitional<BR>> > > G_handle_trans_memb:<BR>> > >
G_handle_trans_memb in GOP<BR>> > > G_handle_trans_memb: Received trans
memb id of: {proc_id: -1408236782<BR>> > > time: 1113940766}<BR>>
> > Memb_regular<BR>> > > Membership id is ( -1408236782,
1113940767)<BR>> > > --------------------<BR>> > >
Configuration at testsys8 is:<BR>> > > Num Segments 1<BR>> >
>
1
239.16.3.18 4803<BR>> >
>
testsys8
172.16.3.18<BR>> > > ====================<BR>> > >
G_handle_reg_memb: with (172.16.3.18, 1113940767) id<BR>> > >
G_handle_reg_memb in GTRANS<BR>> > > G_handle_reg_memb: skipping state
transfer for group RTestGroup.<BR>> > > G_handle_reg_memb: skipping
state transfer for group TTestGroup.<BR>> > > G_handle_reg_memb:
skipping state transfer for group GTestGroup.<BR>> > > Sess_read:
Message has type field 0x800000c4<BR>> > > Sess_read: queueing message
of type 4 with len 0 to the protocol<BR>> > > Sess_read: Message has
type field 0x800000c4<BR>> > > Sess_read: queueing message of type 4
with len 0 to the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > ******** start to receive membership messages
******<BR>> > > received TRANSITIONAL membership for group
RTestGroup<BR>> > > Received REGULAR membership for group RTestGroup
with 1 members, where I<BR>> > > am member 0:<BR>> >
> #RTEST0#testsys8<BR>>
> > grp id is -1408236782 1113940767 1<BR>> > > Due to NETWORK
change. VS set has 1 members:<BR>> >
> #RTEST0#testsys8<BR>>
> > received TRANSITIONAL membership for group TTestGroup<BR>> >
> received TRANSITIONAL membership for group GTestGroup<BR>> > >
Received REGULAR membership for group TTestGroup with 1 members, where I<BR>>
> > am member 0:<BR>> >
> #TTEST0#testsys8<BR>>
> > grp id is -1408236782 1113940767 1<BR>> > > Due to NETWORK
change. VS set has 1 members:<BR>> >
> #TTEST0#testsys8<BR>>
> > Received REGULAR membership for group GTestGroup with 1 members, where
I<BR>> > > am member 0:<BR>> >
> #TTEST0#testsys8<BR>>
> > grp id is -1408236782 1113940767 1<BR>> > > Due to NETWORK
change. VS set has 1 members:<BR>> >
> #TTEST0#testsys8<BR>>
> > ***** we thought things were ok *******<BR>> > > Sess_read:
Message has type field 0x800000c4<BR>> > > Sess_read: queueing message
of type 4 with len 0 to the protocol<BR>> > > Sess_read: Message has
type field 0x800000c4<BR>> > > Sess_read: queueing message of type 4
with len 0 to the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > > Sess_read: Message has type field
0x800000c4<BR>> > > Sess_read: queueing message of type 4 with len 0 to
the protocol<BR>> > ><BR>> > > ****** Something isn't
right ************<BR>> > > Sess_read: Message has type field
0x80000080<BR>> > > Sess_validate_read_header: Message has illegal type
field 0x80000080<BR>> > > SP_error: (-8) Connection closed by
spread<BR>> > > Sess_kill: killing session RTEST0 ( mailbox 14
)<BR>> > > Sess_read: Message has type field 0x800000c4<BR>> >
> Sess_read: queueing message of type 4 with len 0 to the protocol<BR>>
> > Sess_read: Message has type field 0x800000c4<BR>> > >
Sess_read: queueing message of type 4 with len 0 to the protocol<BR>> >
> Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > >
Sess_read: Message has type field 0x800000c4<BR>> > > Sess_read:
queueing message of type 4 with len 0 to the protocol<BR>> > ><BR>>
> > _______________________________________________<BR>> > >
Spread-users mailing list<BR>> > >
Spread-users@lists.spread.org<BR>> > > <A
href="http://lists.spread.org/mailman/listinfo/spread-users">http://lists.spread.org/mailman/listinfo/spread-users</A><BR>>
> ><BR>><BR>><BR>>
_______________________________________________<BR>> Spread-users mailing
list<BR>> Spread-users@lists.spread.org<BR>> <A
href="http://lists.spread.org/mailman/listinfo/spread-users">http://lists.spread.org/mailman/listinfo/spread-users</A><BR><BR>--<BR>-------------------------------------------------------<BR>Jonathan
R. Stanton
jonathan@cs.jhu.edu<BR>Dept. of Computer Science <BR>Johns Hopkins
University <BR>-------------------------------------------------------<BR></FONT></P></DIV>
</BODY>
</HTML>