[Spread-users] Unable to support more than 1000 sessions after increasing MAX_FD_EVENTS
rodrick.brown at gmail.com
Sun Aug 24 17:42:11 EDT 2008
I'm using the Java spread API and connecting to the local spread
daemon version 4.0. Spread was compiled on a 64bit Linux host.
Under heavy load my application stops accepting new messages. I see
about ~1000 connections to local spread daemon before my application
throws a SocketException - Socket(): java.net.SocketException: Too
many open files
On the host running the spread daemon when looking at the number of
local connections to the daemon using lsof I see
spread 16046 spread 10u IPv4 298424
TCP 127.0.0.1:4903->127.0.0.1:35585 (ESTABLISHED)
spread 16046 spread 1013u IPv4 304225
TCP 127.0.0.1:4903->127.0.0.1:49967 (ESTABLISHED)
[root at nybmlx01a logs]# lsof -n |grep -ic ^spread
1076 <<--- Total number of connections to localhost via spread.
So it seems my java process has made about ~1000 concurrent
connections to the local daemon.
I've slightly modified the spread sources and changed MAX_FD_EVENTS in
sp_events.h to 10005 so that we can support 5000 concurrent sessions.
This does not seem to make any difference at all and once again my app
stops responding to new requests around ~1000 concurrent connections.
Is this some kind of bug? I'm not really sure what's going on here. .
I've also tried using the value 55 in MAX_FD_EVENTS and this does seem
to work as expected after about 25 or so connections I'm no longer
able to create any more sessions. What is strange here is that the
error raised is connection refused instead of the Too many open files
error as show before.
I don't think this is a ulimit problem on the Unix host as I'm
starting with the following parameter ulimit -n 60000 in
/etc/init.d/spread which starts spread daemon as root before it
switches user to a non privilege account called 'spread'
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
max nice (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 139264
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 60000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
max rt priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 139264
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Any help will be greatly appreciated thanks.
[ Rodrick R. Brown ]
More information about the Spread-users