[Spread-users] how spread persist messages

[John Song]

It is doable.  One option is to have a message aggregator that listens for
messages from multiple message clients.  The aggregator can do message
persistence.  A side benefit of an aggregator is in preserving orders across
distributed message clients.   Persistence is done to local file system.
Then there is a message distributor daemon that actually distribute the
message to actual message consumers who listens on a different group.

While this method works, there are a couple of reason why it is not ideal:
1. if the aggregator is down, message is lost(?).  For asynchronous message
delivery, ideally, a success return code means that message has been
persisted (not necessarily delivered).  If the local spread client can do
the persistence, then there is less of a chance of dropped messages.
Pulling the existence of an aggregator on every message delivery is both
expensive and still not 100% sure of persistence.
2. the aggregator that persists message need to synchronous with the
dsitributor daemon while reading/writing to the local file system.  While it
is not that difficult, it will be better it is not implemented case by case,
but rather by a component of a mesaging platform.

I read the spread user manual.  The user manual is more a spread
introduction literature plus an api reference manual. I can't find more
indepth architecture discussion on spread's internal.  E.g. spread will
retry a message delivery if there is a network failure.  That means it
maintains an internal buffere.  However, how big is the buffer and how large
each message can be?

Is there any white paper on spread?  Any related projects that has done file
system based message persistence with reader/writer synchronization?

thanks,

-john

On Sat, Mar 15, 2008 at 9:13 AM, Rodrick Brown <rodrick.brown at gmail.com>
wrote:

> On Sat, Mar 15, 2008 at 5:47 AM, John Song <delphi329 at gmail.com> wrote:
> > Hi:
> >
> > I am looking to build a message relay system using spread.  One of the
> > requirement is being able to replay messages in order.  One way is to
> have a
> > listener subscribe to the group and persist messages on disc.  Later on,
> if
> > replay is needed, then a spread client can read the message from the
> disc
> > and resent.  The problem is I can't guarantee the listener will always
> > alive.  Does spread internally persist messages?  I.e. if a client sends
> a
> > message to the group and there is no subscribed listener, will spread
> simply
> > discard the message?
> >
> > Any other open source projects, except JMS, that will be better for that
> > kind of relay/replay system?
> >
> > thanks,
> >
> >
>
> Why not implement multiple listeners? And have them both backup each
> other you can see when other members join/leave groups so it should be
> fairly easy to setup some kind of fail over scenerio where you can
> keep the messages persisted using multiple listeners
>
> > _______________________________________________
> >  Spread-users mailing list
> >  Spread-users at lists.spread.org
> >  http://lists.spread.org/mailman/listinfo/spread-users
> >
> >
>
>
>
> --
> Rodrick R. Brown
> http://www.rodrickbrown.com
> http://www.linkedin.com/in/rodrickbrown
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.spread.org/pipermail/spread-users/attachments/20080315/af205de4/attachment.html