[Spread-users] sendning large messages over spread

Thu Jun 20 03:58:59 EDT 2002

Hi John,

Thank you for your input. I have a c++ interface to spread that people are
currently using and (dare I say) like (It is more than just a wrapper).
Now,
they want to send larger messages (than 100K limit) and they don't
want to do their own "application level" message assembly. So I have said
I may implement it if it does not degrade the performance too much.
So, to cut a long story short, YES PLEASE! Can I look at your Java object?

Another question (not really a spread one):
I have implemented a job scheduler (using my c++ interface to spread)
that invokes jobs (on a solaris machine) according to some rules (specified
in XML & received over spread). My interface uses pthreads heavily
(or lightly, I should say - each job entry is taken care of in a new
thread).
This scheduler works fine when attached to a terminal
but does not when daemonised (blocked). No thread is even invoked before
the
 process is turned into a daemon. Any clues? You can ignore this question
if you like since it is not really a spread question.  Where in you
multi-threaded spread
version do you  turn your process into a daemon. Just in case you are
interested, here is how I turn my scheduler into a daemon:

void CHWScheduler::Daemonise(const CString& daemonName, int facility=0)
        {
        int i;
        pid_t pid;

        if ((pid = fork()) != 0) // makes sure process is run in the
background
                exit(0);

        setsid(); // becomes session leader

        signal(SIGHUP, SIG_IGN);

        if ((pid = fork()) != 0) // enusres we cannot acquire a controlling
terminal
                exit(0);

        chdir("/"); // makes sure we are not on a file system that can be
unmounted
        umask(0); // makes sure inherited files do not affect permission
settings on new ones

        // close all inherited descriptors

        for (int i = 0; i < MAXFD; i++) // MAXFD is softly hard coded to 64
:-)
                close(i);

        openlog(daemonName, LOG_PID, facility); // log to syslog
        }

I have taken this code out of the "W. Richard Stevens" book Network
Programming, Volume 1
2nd edition page 336. The original c routine was called daemon_init.

At some point I would like to make this c++ implementation (which does flow
control and persistence using Berkeley DB) available to anyone who is
interested
(conditional to the permission of the company I am currently consulting
with).

Best Regards,

Koorosh

                    John Schultz                                                                                                  
                    <jschultz at d-f        To:     koorosh.alahiari at ids.allianz.com                                                 
                    usion.net>           cc:                                                                                      
                                         Subject:     Re: [Spread-users] sendning large messages over spread                      
                    19/06/02                                                                                                      
                    18:32                                                                                                         

Spread does not allow for arbitrarily sized messages. It defines a
maximum msg size, where the default is something like 100KB. I don't
know whether or not this limit is publicly exported by Spread currently.
I believe there are checks in the client lib and the daemon on clients'
msg sizes. If a message is too large, then the send should be denied by
Spread.

The DROP_RECV semantics were added to help receivers be able to handle
msgs larger than they expected. Previously in Spread, if you called recv
with too small buffers your buffers would be filled and the excess would
be discarded. This isn't the best semantics for reliable communication
:). So the default was changed such that if the receiver's buffers were
too small, then the recv would fail while returning information about
how big the receiver's buffers needed to be. If the caller doesn't
resize their buffers or use DROP_RECV then the recv will continue to fail.

If you call recv with DROP_RECV, you are electing to have the old
semantics where the recv will succeed but some of the information in
that message may be dropped if the msg is larger than your buffers. This
was added so that program logic could be simpler if the application
author so chose.

A while back I posted an extension to the Java SpreadConnection that
allowed for arbitrarily sized messages, while degrading some of the
safety semantics of Spread. You can look at the archives or the website
to find it, or if you want I'll send it to you. This would give you a
good place to start to implement your own, or you could just use it if
you are using Java.

--
John Schultz
Co-Founder, Lead Engineer
D-Fusion, Inc. (http://www.d-fusion.net)
Phn: 443-838-2200 Fax: 707-885-1055

koorosh.alahiari at ids.allianz.com wrote:

> Hi All,
>
> As I understand it you can send arbitrarily large(within reason)
messages
> to
> spread and spread divides them into its max 80K limit chunks and delivers
> them all. Receiving a service type of DROP_RECV will indicate that there
is
> more chunks left for this message. So calling SP_receive repeatedly
> until the the service type is no longer DROP_RECV  gurantees that I can
> assemble the original large message without having to resort to a higher
> level
> protocol. Is this correct. Is it guranteed that I will not get any other
> messages
> (in between) until I have read the entire large message?
>
> Hope it makes sense!
>
> Regards, Koorosh
>
>
>
> _______________________________________________
> Spread-users mailing list
> Spread-users at lists.spread.org
> http://lists.spread.org/mailman/listinfo/spread-users
>
>
>