[Spread-users] Question about spread v4.1

[John Schultz]

Other configurations you could try would be:

(A) 1 Spread daemon, with 2 spread flooders running on the same machine.
(B) 2 Spread daemons on different machines, each with a spread flooder running locally.

Yair's comment was that in your configuration (and in the above configuration (A)) there is only one Spread daemon, so all it is essentially acting as is a hub for client communications.  There are no issues of flow control, which are the tuning parameters to which he was referring, because there is only one network entity: the single daemon.

All of that being said, you need to make sure that you are looking fairly at the results you get.  In your first configuration, the Spread daemon was handling 660 Mb/s of I/O, which is decent for a single threaded, event driven application.

Spread will never be able to compete with native UDP or TCP when it comes to throughput in ultra fast networks because it acts as a user-level middle man and consequently has more OS context switches in the messaging path and higher CPU load.  I also don't believe Spread has ever been extensively profiled to optimize its computational hot spots.  Spread's primary concern is about giving powerful semantics to users that makes distributed programming easier.  Performance in throughput and latency are very important, but they are not the primary goal of Spread.

Cheers!

-----
John Lane Schultz
Spread Concepts LLC
Phn: 301 830 8100
Cell: 443 838 2200

On Feb 18, 2011, at 7:16 AM, Parisa wrote:

Thanks John for the response. 

I am gonna check the CPU usage of daemon to see if it is the bottleneck. In this case do you think by adding more Daemons I can get a better performance (maybe on different machines). Because my goal here is to just get the best of spread, no matter what is the configuration. I have sent that same email to Prof. Yair. Amir and he wrote me that: 

"
There are ways to tune the Spread toolkit for performance, using
the spmonitor, but they will not affect your setup, which only has a
single daemon that serves two remote clients.
In effect, your setup is not really using Spread for messaging.

The Spread toolkit has a mailing list to which you can post questions
at www.spread.org. If you have further questions, I suggest you
post them there.


"

I am not sure if I quite understand this. So that would be great if you could help with spmonitor and how can I truly use spread for messaging. 


Thanks 

Parisa. 



On Feb 17, 2011, at 7:58 PM, John Schultz wrote:

> Parisa,
> 
> I don't know any quick fixes that will increase your throughput.  Most likely, the machine on which your Spread daemon is running is CPU throttled (i.e. - it's using 100% of CPU).
> 
> If I understand your setup correctly, a 220 Mb/s TCP stream flows from the sending flooder to the Spread daemon and then two TCP streams of 220 Mb/s each flow back to the two flooders.  The single threaded Spread daemon is handling about 660 Mb/s of I/O, which isn't too shabby.
> 
> I'd be interested to know if the bottleneck was your sending flooder or the Spread daemon.  Regardless, Spread is pretty CPU intensive and to improve it you'd likely need to do an in depth profiling and optimization.
> 
> Cheers!
> 
> -----
> John Lane Schultz
> Spread Concepts LLC
> Phn: 301 830 8100
> Cell: 443 838 2200
> 
> On Feb 15, 2011, at 11:51 AM, Parisa wrote:
> 
> 
> Hello, 
> 
> I have been running spread version 4.1 with the following settings:
> 
> 1- I used examples/flooder.c code augmented with some functions to measure the throughput. I am attaching the modified file here. 
> 2- 16000 bytes are specified for each packet. 
> 3- There is one daemon in a separate machine and 2 flooders in two separate machines. 
> 4- Machines are connected via a 1Gb network and are running Linux Ubuntu . 
> 
> after running the flooder/spread for 55 seconds the throughput obtained with the setting explained is 220 Mb/s. 
> 
> Parameters passed to the spflooder are specified as:
> 
> 	-u #id -s 4803 at spread_machine -b 16000  -n 2 -m 10000
> 
> By decreasing/increasing the packet size, or increasing the number of flooders throughput reduces. 
> 
> I was wondering if there is another setting that can lead to a higher throughput. As this result is  obtained in a 1 Gb network. 
> 
> Thanks.
> 
> 
> <flooder.c>
> 
> 
> ------------------------------------
> Parisa Jalili Marandi
> 
> PhD Student
> Faculty of Informatics
> University of Lugano
> via G.Buffi 13
> CH-6904 Lugano - Switzerland
> 
> www.inf.usi.ch/phd/jalili
> 
> 
> 
> 
> 
> 
> 





-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3805 bytes
Desc: not available
Url : http://lists.spread.org/pipermail/spread-users/attachments/20110218/59775141/attachment.bin