[Spread-users] Will Spread work for me - large numbers of remote locations connected via satellite internet

Yair Amir yairamir at cnds.jhu.edu
Thu Mar 4 05:30:14 EST 2004

Hi Terver,

Your environment is very interesting. I have started to look at
such environments myself (actually, a bit more complicated then
what you describe).

Unfortunately, Spread will not meet your need.

In my opinion, your intuition is correct and the best solution will
be, in concept, something like a modified hop protocol between your
clients and the server. The problem is challenging because of the
very high latency, a probable relatively-high loss rate, and the
high number of clients.
I am sure it can be done with a custom protocol.


       :) Yair.   http://www.cs.jhu.edu/~yairamir
On Thursday, March 04, 2004 1:21 AM
Trever Miller trever at cyberdex.ca wrote:

Trever> Good day,

Trever> I am interested in using spread but am unsure if it will fit my environment.

Trever> Summary of the problem space, followed by questions

Trever> Embedded oil field data capture systems (SCADA) in remote locations
Trever> connected to the internet via satellite modem service.  The satellite
Trever> modem and embedded data collection/transmission system are on the
Trever> internet, pingable, and have 1200-3000 ms latency as measured by ping.
Trever> The size of each data payload is quite small, under 20K, and is usually
Trever> generated hourly.

Trever> We use SSH to connect to the systems for diagnostics and to push updated
Trever> configurations and programs to the embedded units.   The data
Trever> transmission uses scp driven by shell scripts that play games with
Trever> secondary files that contain CRC and other info, in an attempt to ensure
Trever> that only full complete data report files make the trip.  Validation
Trever> happens on the receiving end in the data center.

Trever> The scp based data transport is already having issues, with several
Trever> retries being the norm as the ssh connection fails quite often in the
Trever> handshake/setup phase due to lost or extremely delayed tcp packets.
Trever> The data does eventally get thru, and the remote sites do manage to also
Trever> pick up their configuration changes or program updates from the
Trever> mothership.   I am not comfortable with the amount of failed and
Trever> restarted connections in the original 30 trial sites though.

Trever> The number of sites is going to grow to 100 soon, and many hundreds
Trever> later this year.  Ultimately in the thousands.

Trever> I want to use spread as the new data transport mechanism, and also to
Trever> push configuration data, program updates out.

Trever> I think I will be using the private group thing for talking to
Trever> individual sites, and the sites would talk to a small number of groups
Trever> hosted at the server side, where data would be grabbed for processing
Trever> and insertion into the application that needs the data.

Trever> Here's my questions.  Thanks for reading thru all of the above.

Trever> The hop protocol, being based on udp, looks like a good fit here.
Trever> Unfortunately my understanding of how spread works, is that the hop
Trever> protocol is only used between spread daemons.  Meaning I would have to
Trever> put a daemon on each of the embedded units.  OK, I can do that, no
Trever> problem.   But... other stuff in the docs and on this list indicate that
Trever> there is a hard upper limit of 128 daemons.

Trever> Is there something I am missing?

Trever> Can the client libs use hop to talk over the high latency wan to the
Trever> server processes at the data center?

More information about the Spread-users mailing list