[Spread-users] [BUG][PATCH] fd to session troubles

[Tim Peters]

[Jonathan Stanton]
> So you are saying if we add a line
> #define FD_SETSIZE 1024
>
> in the Windows section of our arch.h, then windows select will work with
> 1024 sockets at once?

Basically yes, but I'm afraid it's subtler than that:  if you *don't* add
such a define before including winsock.h, then there's no chance select()
will work correctly with 1024 sockets at once (setting up the fd_set would
corrupt memory beyond the end of the socket handle array, which has only 64
slots by default).  If you do add such a line, it may or may not work
correctly, depending on the socket provider, because:

>just want to verify it because it seems like the windows libc/equivelent
> might have it's own compiled in knowledge of the size of the fd_set
> struct and think it is only 64.

There's the rub:  the socket provider isn't part of the Windows kernel, or
of MS libc, and what you get depends entirely on which socket-providing DLL
you link with.  Different providers can have different limits.  So long as
you're sticking to Winsock version 1, the provider is supposed to set the
maximum # of sockets it can actually handle in the iMaxSockets field of the
WSAData struct passed to WSAStartup().  For Winsock 2, I never did figure
out how you're supposed to know.  In any case, FD_SETSIZE is purely a
client-side #define, and the provider believes whatever fd_count it sees in
an fd_set struct.

In practice, Python uses wsock32.dll as its socket provider, and boosts
FD_SETSIZE to 512 before including winsock.h:

/* Windows #defines FD_SETSIZE to 64 if FD_SETSIZE isn't already defined.
   64 is too small (too many people have bumped into that limit).
   Here we boost it.
   Users who want even more than the boosted limit should #define
   FD_SETSIZE higher before this; e.g., via compiler /D switch.
*/
#if defined(MS_WINDOWS) && !defined(FD_SETSIZE)
#define FD_SETSIZE 512
#endif

...

#ifdef MS_WINDOWS
#include <winsock.h>
#else
#ifdef __BEOS__
#include <net/socket.h>
#define SOCKET int
#else
#define SOCKET int
#endif
#endif

and no problems have been reported even on Win95.  I expect we could have
boosted it to 1024 without troubles too, but there are tradeoffs, and users
for whom 512 isn't enough are generally doing "researchy things" and want to
try tens of thousands.

What you should do is rewrite Spread in Python, and then you'll get a
platform-independet socket API for free <wink>.