FW: [Spread-users] Two daemons per machine? Two ports per daemon?

[Killeen, Damien]

The main reason we want to do this is to keep aside ALL of the infiniband network's bandwidth for the high-rate comms and not waste any of it on the low-rate comms.  I did try a method similar to your suggestion early on, but made the mistake of using concurrent port numbers!

I have now tried implementing your scheme, with the following exception.  I have been advised that a.b.c.255 and 192.168.1.255 are the broadcast addresses that I should use for our network set up, so my config's are:

	Spread_Segment a.b.c.225:4803 { # Here, your suggestion was 255.255.255.255:4803
		machine1	a.b.c.1
		machine2	a.b.c.2
	}
and:
	Spread_Segment 192.168.1.255:5803 { # Here, your suggestion was 255.255.100.100:4803
		machine1	192.168.1.1
		machine2	192.168.1.2
	}

Not being entirely knowledgeable about IP, I do not know whether making this change to your suggestion is what might be causing the following problem:

When I use the high-rate config for the infiniband NIC's IP, the daemon fails to load because "my proc id (a.b.c.d) is not in congfiguration" (where a.b.c.d is the host IP which is different to the infiniband NIC's IP).  It seems that the daemon resolves the host IP based on host name only instead of the entire host.domain name.


> -----Original Message-----
> From: spread-users-admin at lists.spread.org
> [mailto:spread-users-admin at lists.spread.org]On Behalf Of John Schultz
> Sent: Saturday, 22 May 2004 12:32 AM
> To: spread-users at lists.spread.org
> Subject: Re: [Spread-users] Two daemons per machine? Two ports per
> daemon?
> 
> 
> An important question would be why do you want to do this and do you 
> really need to do it?  If you email your reasoning, the list 
> may be able 
> to advise you on if this is a good approach or not.
> 
> I'm assuming you want to keep the two streams of data separate.
> 
> You can do this simply by having two configuration files: one 
> specifying 
> the low rate network and the other specifying the high rate network. 
> Your application would connect to both of the daemons (SP_connect to 
> "4803" and "5803") and your code would have to condition on which 
> connection you send on depending on if you want to send low 
> rate or high 
> rate data.
> 
> If you ensure that the login name that use in SP_connect is 
> unique per 
> instance of your application and you keep the daemon names 
> the same for 
> each machine, then the private names given to you by the two daemons 
> should be the same.
> 
> For example:
> 
> low-rate.conf
> 
> Spread_Segment 225.225.225.225:4803 {
> 	machine1	a.b.c.1  # fill in your real IP info.
> 	machine2	a.b.c.2
> }
> 
> ...
> 
> high-rate.conf
> 
> Spread_Segment 225.225.100.100:5803 {
> 	machine1	x.y.z.1  # fill in your real IP info.
> 	machine2	x.y.z.2
> }
> 
> ...
> 
> This all assumes that you can somehow force the OS to map the two 
> multicast/broadcast addresses/ports to the two different 
> cards, so that 
> the low rate configuration runs on the ethernet card and the 
> high rate 
> configuration runs on the Infiniband.  You may be forced to use 
> broadcast addresses (I assume the two cards would have distinct 
> broadcast addresses) if you can't figure out how to force the 
> OS to map 
> multicast addresses to different cards.
> 
> It is important to remember that Spread also uses an extra 
> unicast UDP 
> port one higher than the one you specify in the file bound to the 
> address you associate with the machine.  For example, the low rate 
> daemon on machine1 would bind to UDP multicast/broadcast 
> (225.225.225.225, 4803) AND bind to UDP unicast (a.b.c.1, 4804).  The 
> high rate daemon would do similarly but for the different addresses.
> 
> The bottom line being that you need to make sure both of these 
> address/port pairs map properly to the different cards.
> 
> -- 
> John Lane Schultz
> Spread Concepts LLC
> Phn: 443 838 2200
> 
> Killeen, Damien wrote:
> 
> > Hi All,
> > 
> > I am a relatively new user of spread and am trying to set 
> up a spread network across a cluster of machines running 
> redhat 8, each with both an Ethernet NIC and an Infiniband 
> NIC (to use spread over infiniband I will need to add an IP 
> layer unless the spread development team has plans to support 
> other-than-IP based networks in the future?).  I plan to use 
> the ethernet connection for low-rate comms and the infiniband 
> NIC for very-high-rate data.
> > 
> > I would like our applications to connect to the daemon 
> using, for example, port 4803 to specify the comms link and 
> 5803 to specify the data link, and not have to specify the 
> actual IP of either connection in the code (so that we can 
> run the binaries anywhere on the cluster - "location 
> transparency" is one of the greatest benefits of group 
> communications for our project).
> > 
> > I can only see this being possible by either running two 
> daemons on each machine (configured separately with the 
> different port numbers, one for comms and one for data), or 
> by having two ports to connect to the daemon.  The former 
> solution is preferred, as it would allow our applications to 
> connect to both ports/daemons with the same private name.  
> Are either of these solutions possible, and if so, how?
> > 
> > Thanking you in advance,
> > 
> > Damien Killeen.
> > 
> > _______________________________________________
> > Spread-users mailing list
> > Spread-users at lists.spread.org
> > http://lists.spread.org/mailman/listinfo/spread-users
> > 
> 
> 
> 
> 
> _______________________________________________
> Spread-users mailing list
> Spread-users at lists.spread.org
> http://lists.spread.org/mailman/listinfo/spread-users
>