[Spread-users] problem with spread/mod_log_spread/spreadlogd
sbarvick at revasystems.com
Wed Sep 7 13:08:55 EDT 2005
Has anyone given any thought to passing daemon or connection congestion
status back through through messages received by the clients? I can
easily kill my Java client with too much traffic, but I might be able to
slow down the traffic if I knew I was starting to get into to trouble.
Having a client close with an error is more disruptive in some cases
that cutting back on the data sent so it would be nice to have that
On Tue, 2005-09-06 at 17:43, John Schultz wrote:
> FYI, Spread by default will buffer up to 1,000 msgs for a connection
> before it will kick it. The thing is that you can push Spread to pass
> over 10,000 (small) msgs per second in a LAN. So if you have some kind of
> extreme msg spike and a reader can't keep up in that time frame, then I
> believe they can be disconnected in less than a second!
> I believe you can raise this limit by modifying spread_params.h
> #define MAX_SESSION_MESSAGES 1000
> try raising it by an order of magnitude
> #define MAX_SESSION_MESSAGES 10000
> Now the potentially negative side effect of this is that your spread
> daemon will probably eat more memory on your system. So you need to
> ensure there is memory for it to grow because you don't want the daemon to
> die due to low memory.
> Good luck!
> John Schultz
> Spread Concepts
> Phn: 443 838 2200
> On Tue, 6 Sep 2005, Jeroen wrote:
> > Theo Schlossnagle wrote:
> >> John Schultz wrote:
> >>> Well -11 is CONNECTION_CLOSED, which just means the connection between the
> >>> client and daemon has been shut down. The most common reason for this is
> >>> a flow control problem where msgs are being injected into the system
> >>> faster than readers can read them out. At some point Spread will kick the
> >>> connection so that it doesn't run out of memory and kill the daemon, thus
> >>> losing all of its connections.
> >>> I'm not familiar with mod_log_spread and I don't know if it performs any
> >>> kind of flow control. If it doesn't and you are logging too fast this
> >>> could cause your clients to be repeatedly disconnected (assuming they
> >>> reconnect).
> >> mod_log_spread does no flow control what-so-ever. spreadlogd will read
> >> message from Spread as fast as it can write to disk. So, the typical
> >> reason for this sort of behaviour is that you try to journal the logs from
> >> your entire cluster on an IDE system or some other slow storage facility.
> >> The lack of flow control was a design decision in mod_log_spread. In otder
> >> to have time-ordered, real-time logs, you either must have no flow control
> >> or you must allow the publishers to block. In m_l_s, it was decided that
> >> under no circumstances should publishers block (as that would mean a
> >> slowdown in serving web pages). If that approach doesn't "jive" with your
> >> idea of logging in a web cluster, then m_l_s isn't for you.
> >> (The *you* above, is of course not John, but whomever is running m_l_s)
> > Thank you for your awnser!
> > this is exactly what my thought were, but the problem is that the system that
> > is writing the logs to disk is a scsi-320 system with hardware raid-1 :)
> > it's just writing about 100mb of logs per hour, that doesn't look that much
> > to me.
> > Load is low on that server since it is a dedicated logserver, but I/O might
> > be a problem (since that isn't always shown in pure load).
> > I'll hookup a raid-10 system with 15k rpm disks tomorrow and test further.
> > Another problem believing this conclusion is that we have another cluster
> > that is performing great loging to a sata-raid array, writing almost 2 gigs
> > of logs every hour, but that is another setup.
> > Kind regards and please give your comments since this isn't solved yet,
> > Jer
> > _______________________________________________
> > Spread-users mailing list
> > Spread-users at lists.spread.org
> > http://lists.spread.org/mailman/listinfo/spread-users
> Spread-users mailing list
> Spread-users at lists.spread.org
More information about the Spread-users