[Spread-users] NegativeArraySizeException under load

[toby cabot]

Hi Folks,

Spread's cool, thanks!

We're seeing problems with the Java driver when we send a reasonable
(greater than a couple hundred messages per second) load of messages
across Spread.  At least one other person [1] has seen it too, but I
can't find much info in the list archives so hopefully this data will
help.  We get exceptions like

java.lang.NegativeArraySizeException
  at spread.SpreadConnection.internal_receive(SpreadConnection.java:1195)
  at spread.SpreadConnection.access$200(SpreadConnection.java:70)
  at spread.SpreadConnection$Listener.run(SpreadConnection.java:1663)

the relevant code is

         // Read in the group names.
         ///////////////////////////
1195==> byte buffer[] = new byte[numGroups * MAX_GROUP_NAME];
         try
         {
               for(int bufferIndex = 0 ; bufferIndex < buffer.length ; )

That seems odd since there's a check a few lines above 1195 to make sure
that numGroups > 0.  But in our case numGroup's value was so large that
when it was multiplied by MAX_GROUP_NAME it "wrapped around" and became
negative, thus tripping the exception.  So question 1 is should we add
another check to make sure that numGroups > 0 *and*
numGroups*MAX_GROUP_NAME > 0?

I was only sending to a single group so I was surprised that the value
of numGroups was so large so I looked at it and when I converted it to
characters I could see fragments of the payload of my message (lucky for
me some of my messages are ASCII otherwise it would have been much
harder to see).  Evidently the driver is missing the header and
interpreting message payload data as header data.  So question 2 is
should the message headers have framing data so that the driver can tell
when this happens?  I think it would be a good thing since I can imagine
cases where numGroups or other fields could be wrong but pass basic
sanity tests, which means that the driver could deliver garbage to the
application.  Of course this is a protocol change so it would require a
non-trivial effort to implement.

Question 3 is the most important: why is the driver missing the header?
  I'm using the AdvancedMessageListener interface and my callback
doesn't do any real work, it just queues the message and notifies
another thread to handle it.  So I don't think that it could be much
more efficient than it is now, but for some reason the driver doesn't
get back to the read() in time to read the header.  Any thoughts?

Thanks,
Toby

[1] nilay padh, Fri, 25 Jun 2004
http://lists.cnds.jhu.edu/pipermail/spread-users/2004-June/002087.html