cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheng Yang <sh...@yasker.org>
Subject Re: Working on CloudStack Jira-764:nTier Apps 2.0 : Redundant Virtual Router for VPC email 2 of 2
Date Mon, 03 Feb 2014 19:50:05 GMT
On Mon, Feb 3, 2014 at 11:27 AM, Karl Harris <karl.harris@sungard.com>wrote:

> On Mon, Feb 3, 2014 at 2:03 PM, Daan Hoogland <daan.hoogland@gmail.com
> >wrote:
>
> >
> > On 03 Feb 2014, at 19:45, Karl Harris <karl.harris@sungard.com> wrote:
> >
> > > On Mon, Feb 3, 2014 at 12:54 PM, Daan Hoogland <
> daan.hoogland@gmail.com
> > >wrote:
> > >
> > >>
> > >> On 03 Feb 2014, at 18:03, Karl Harris <karl.harris@sungard.com>
> wrote:
> > >>
> > >>> After discussion with my colleagues  questions about initial
> > >>> configuration of the open network redundant routers and the
> > >>> applicability of the existing bash scripts (cloud-early-config) to
> the
> > >>> setup of VPC redundant routers have generated.
> > >>>
> > >>>
> > >>> Some setup first: In the bash script cloud-early-config there is a
> > >>> function named setup_redundant_router which makes copies of several
> > >>> template files. The template files are used to configure keepalived
> > >>> and conntrackd. The template copies are edited, via sed editor, using
> > >>> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.)  which are
> > >>> obtained from the kernel of the current running linux image using the
> > >>> virtual file system /proc/cmdline.
> > >>>
> > >>> I'm sure keepalived and conntrackd  can be used for starting and
> > >>> control of VPC redundant routers. However the setup of keepalived and
> > >>> conntrackd for VPC needs setup parameters which are dynamic because
a
> > >>> VPC can have N number of redundant router pairs, not just the fixed
> > >>> number parsed from proc/cmdline in the running kernel.
> > >>>
> > >>> Am I correct in this analysis?
> > >> Karl, I think not. There is only one router in a vac, it routes for
> all
> > >> networks in the virtual private cloud. Am I misreading your
> description.
> > >
> > > Ok then a VPC should contain a single router containing N+1 (N+2 if
> > using a
> > > VPN connection) nic's? Where N is the number of private networks.
> > And I would like to see N+3 as it will allocate another when a private
> > gateway is defined. That is why I proposed to pre-allocate this nic to be
> > able to predict mic-ids, i.e. eth<#>.
> >
> > >
> > > If so then my original question still holds in the sense that the nic's
> > > need to be created in the VM kernel based on the number of private
> > networks
> > > desired?
> >
> > Yes
> >
> > > I found the method update_System_Vm_Templates in the jar file
> > > Upgrade410to420 in cloud-engine-schema/src/com/cloud/upgrade/dao which
> > > seems to point to a set of pre-configured VM images.
> > I can not find the method you describe but these preconfigured images are
> > the ones used for all the system vms. That would be secondary storage
> vms,
> > console proxy vms and indeed the router vms.
> >
> > Based on the discussion above the the configuration part of the VPC
> redundant router is:
>
>  1. Collecting the correct data required to configure the nics. (Is this
> part already in the VPC setup code? I will look for it, pointers will be
> helpful.)
>

vpc_guestgw.sh


>  2. Based on the hypervisor/vm types configure the nic(s) on the router VM
> using the data collected above (Same question as in 1 above is this code
> already in place?). My guess is to start with a preconfigured template and

configure the vm based on the number of nics and any other appropriate data.
>

Yes.


>  3. Configuring conntrackd and keepalived after the router vm (nics) have
> been configured and the router(s) started or restarted? I still need to
> work through this.
>

Likely you would need this.

Don't know if keepalived can be start if there is no virtual IP definition.

You also need to think about what to do second router(or the router pair)
when the last tier of VPC has been destroyed.

--Sheng


>
>
>
>
> >
> > >>
> > >>> If so, given the dynamic nature of the VPC redundant router
> > >>> configurations: Is using a setup_VPC_redundant_router bash function,
> > >>> similar to the existing open network function mentioned above, the
> > >>> most appropriate way to setup the keepalived or conntrackd
> > >>> configuration files for VPC redundant routers in the
> > >>> cloud-early-script? It seems to me reading the parameters from the
> > >>> kernel will require a unwieldy set of kernels to match the N private
> > >>> network redundant router pairs configured by the enduser.
> > >>>
> > >>>
> > >>> Comments, questions, clarifications?
> > >>
> > >>>
> > >>> Karl
> > >>>
> > >>>
> > >>> In the bash script ea
> > >>> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland <
> > daan.hoogland@gmail.com>
> > >> wrote:
> > >>>> good reason to skip it for a next version, let's look into it
> anyway,
> > >>>> as we don't want to burn any of our ships.
> > >>>>
> > >>>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris <
> karl.harris@sungard.com
> > >
> > >> wrote:
> > >>>>> All,
> > >>>>>
> > >>>>> At first redundant DHCP seemed like a good idea. I did some
cursory
> > >>>>> research and the more I read the more I'm convinced it may
be
> > >>>>> more trouble than its worth for the first implementation. I'll
talk
> > >>>>> with some of our Systems Engineer's here and get a broader
> > >>>>> perspective.
> > >>>>>
> > >>>>> There seems to be only a single implementation of an open source
> DHCP
> > >>>>> server that will handle the synchronization required for redundant
> > >>>>> servers.
> > >>>>>
> > >>>>> Karl
> > >>>>>
> > >>>>>
> > >>>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland <
> > >> daan.hoogland@gmail.com> wrote:
> > >>>>>> Saurav,
> > >>>>>>
> > >>>>>> Not sure how this happens now, but it is definateluy something
we
> > >>>>>> want. For the static conf files it won't be much of a problem.
The
> > >>>>>> firewall/loadbalences won't be much of a problem, they
are fire
> and
> > >>>>>> forget configurations of the ms that can just be sent to
both. The
> > >>>>>> dhcp is a challange. I am not sure if it is solved for
the plain
> rvr
> > >>>>>> now but it should be solved for that as well.
> > >>>>>>
> > >>>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri
> > >>>>>> <saurav.lahiri@sungard.com> wrote:
> > >>>>>>> Daan,
> > >>>>>>> I was wondering if you had not shared your thoughts,
but looks
> like
> > >> I had
> > >>>>>>> missed your mail.
> > >>>>>>>
> > >>>>>>> I agree that redundant dhcp or dns would be good to
have. What I
> > >> meant was,
> > >>>>>>> it appears that by  just enabling RVR   there is no
way to auto
> > sync
> > >>>>>>> configuration between the  master and slave nodes with
regard to
> > >> dhcp,
> > >>>>>>> loadbalancer and firewall(specifically the dhcp lease
file,
> > >> haproxy,cfg and
> > >>>>>>> iptables configuration).  So just enabling RVR does
not ensure
> high
> > >>>>>>> availability for  these services. Is there a way cloudstack
> > autosyncs
> > >>>>>>> configuration?
> > >>>>>>>
> > >>>>>>> For the routing portion this is not an issue as the
participating
> > >> routers
> > >>>>>>> learn the route through known protocols.
> > >>>>>>>
> > >>>>>>> Thanks
> > >>>>>>> Saurav
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan Hoogland <
> > >> daan.hoogland@gmail.com>wrote:
> > >>>>>>>
> > >>>>>>>> Saurav, I don't see why you can't benefit from
having other
> > services
> > >>>>>>>> redundant as well. Vpn might be a problem as the
source ip of a
> > >>>>>>>> redundant router pair on a vpn might give a problem
(maybe there
> > is
> > >> an
> > >>>>>>>> implementation) but why wouldn't you want redundant
dhcp or dns
> > >>>>>>>> services? As I understand it these are used at
Schuberg Philis
> at
> > >> the
> > >>>>>>>> moment. will double check when I get a chance.
> > >>>>>>>>
> > >>>>>>>> regards,
> > >>>>>>>> Daan
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav Lahiri
> > >>>>>>>> <saurav.lahiri@sungard.com> wrote:
> > >>>>>>>>> Daan,
> > >>>>>>>>> So looking at what is available today for guest
network, the
> > >> Redundant VR
> > >>>>>>>>> can be enabled only for the source nat service.
I guess the
> > >> mention of
> > >>>>>>>> the
> > >>>>>>>>> source nat router is in relation to that. I
could be wrong
> > though.
> > >> It
> > >>>>>>>>> appears  that the other services like vpn,
dhcp, dns do not
> > >> benefit much
> > >>>>>>>>> from the RVR capability. Can you clarify if
thats correct?
> > >>>>>>>>>
> > >>>>>>>>> Thanks
> > >>>>>>>>> Saurav
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM, Karl Harris
<
> > >> karl.harris@sungard.com
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> ---------- Forwarded message ----------
> > >>>>>>>>>> From: Daan Hoogland <daan.hoogland@gmail.com>
> > >>>>>>>>>> Date: Wed, Jan 15, 2014 at 2:51 PM
> > >>>>>>>>>> Subject: Re: rvr4vpc
> > >>>>>>>>>> To: Karl Harris <karl.harris@sungard.com>
> > >>>>>>>>>> Cc: Christopher Litsinger <christopher.litsinge@sungard.com>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> H Karl,
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks for sharing. I didn't want to initiate
this but I
> > >> encourage you
> > >>>>>>>>>> to share this on the dev list (not in jira)
as things are only
> > >>>>>>>>>> considered 'discussed' if they passed by
there.
> > >>>>>>>>>>
> > >>>>>>>>>> You speak of '1 Get configuration data
on Source Nat Router',
> I
> > >> don't
> > >>>>>>>>>> understand why you call the router by this
designation.
> 'Source
> > >> Nat'
> > >>>>>>>>>> is only one of it's many possible functions.
> > >>>>>>>>>>
> > >>>>>>>>>> Apart from the design principles I shared
with you I have so
> far
> > >> only
> > >>>>>>>>>> a technical implementation detail so far.
That is to reserve
> the
> > >>>>>>>>>> (eth2) interface for the private gateway
on the vpc (r)vr.
> This
> > >> way
> > >>>>>>>>>> the inteface to configure are somewhat
predictable.
> > >>>>>>>>>>
> > >>>>>>>>>> As for the design principle to have a statefull
router (reboot
> > >> proof)
> > >>>>>>>>>> the idea is to implement a configuration
file that will be
> > >> uploaded to
> > >>>>>>>>>> the router after which a self-config command
is send that will
> > >>>>>>>>>> implement the details of configuring the
interfaces, haproxy
> and
> > >>>>>>>>>> keepalived and maybe more. I think your
current assessment of
> > the
> > >>>>>>>>>> working of the RVRs is accurate but it
will not be workable
> for
> > an
> > >>>>>>>>>> implementation for vpc's as they have an
unpreditable number
> of
> > >>>>>>>>>> interfaces.
> > >>>>>>>>>>
> > >>>>>>>>>> to bad you can't make it next thursday,
> > >>>>>>>>>> Daan Hoogland
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, Jan 15, 2014 at 3:25 PM, Karl Harris
<
> > >> karl.harris@sungard.com>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>> Daan,
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sorry for the delayed response but
as Christopher mentioned
> to
> > >> you in
> > >>>>>>>> his
> > >>>>>>>>>>> email I am getting my head around the
CloudStack software.
> > >>>>>>>>>>> Since I am new to CloudStack but "old"
to enterprise level
> JAVA
> > >> the
> > >>>>>>>> task
> > >>>>>>>>>> is
> > >>>>>>>>>>> large but not impossible. I have no
experience with running
> > >> CloudStack
> > >>>>>>>>>> but
> > >>>>>>>>>>> considerable experience designing and
maintaining large JAVA
> > >>>>>>>>>> applications.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I've created what I believe is a very
high level abstract of
> > how
> > >> the
> > >>>>>>>>>> current
> > >>>>>>>>>>> guest VRR's are created for guest networks
with the intent of
> > >> making
> > >>>>>>>> this
> > >>>>>>>>>>> abstract
> > >>>>>>>>>>> more detailed.
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1 Get configuration data on Source
Nat Router  selected as a
> > >> redundant
> > >>>>>>>>>>> router
> > >>>>>>>>>>>  1.1 Public and Guest network identified.
> > >>>>>>>>>>> 2 Both routers are provisioned
> > >>>>>>>>>>>  2.1 Software  trys different,
> > >> regions(?),zones,pods,clusters,hosts
> > >>>>>>>> in
> > >>>>>>>>>>> that order as the location of the router.
Log maximum
> > "distance"
> > >>>>>>>> apart.
> > >>>>>>>>>>> 3 Keepalived is configured
> > >>>>>>>>>>> 4 Both routers are started
> > >>>>>>>>>>> 5 Keepalived is started
> > >>>>>>>>>>>
> > >>>>>>>>>>> Obviously there is much more that is
happening under each of
> > the
> > >> steps
> > >>>>>>>>>>> above. My intent is to complete this
detailed "as is" listing
> > as
> > >> much
> > >>>>>>>> as
> > >>>>>>>>>> we
> > >>>>>>>>>>> can. Then  using the "as is" description/sequence
> > >>>>>>>>>>> make a "to-be" addition for VPC's.
When I get a consensus on
> > WHAT
> > >>>>>>>> needs
> > >>>>>>>>>> to
> > >>>>>>>>>>> be implemented for the VRR in VPC 
then develop HOW best to
> > >> implement
> > >>>>>>>> the
> > >>>>>>>>>>> "to-be" addition with the
> > >>>>>>>>>>> existing JAVA code as well as what
additional classes need to
> > be
> > >>>>>>>>>>> extended/implemented/created.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Comments, critiques and changes to
the above sequence are
> > >> encouraged.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I plan on posting this to the dev-list/Jira
very soon.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I have been using this functional spec
as a guide, after
> > >> discussing
> > >>>>>>>> this
> > >>>>>>>>>>> with our Systems Engineering people
this spec meets our
> > >> requirements.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Do you have an implementation in mind?
> > >>>>>>>>>>>
> > >>>>>>>>>>> We have an internal Cloud Meeting with
conflicts with the
> > >> cloudstack
> > >>>>>>>>>> "day"
> > >>>>>>>>>>> next week so I will not be in attendence.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Karl
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Jan 14, 2014 at 4:35 PM, Daan
Hoogland <
> > >>>>>>>> daan.hoogland@gmail.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Hello overthere in the states,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Tomorow I will start some experimenting
with redundant vpc
> > >> routers.
> > >>>>>>>>>>>> This is to check up on any findings
and requirements that
> you
> > >> might
> > >>>>>>>>>>>> have on this. Once again I would
not like to waste work on
> > this
> > >> as it
> > >>>>>>>>>>>> is really a globally usable feature
that is probably
> > universal.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> please let me know your status
on this.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> If any of you are coming to the
cloudstack day in London
> next
> > >> week,
> > >>>>>>>>>>>> let's meetup next thursday.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> kind regards,
> > >>>>>>>>>>>> Daan Hoogland
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> Karl O. Harris
> > >>>>>>>>>>> Cloud Software Engineer
> > >>>>>>>>>>> Sungard Availability Services
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> Karl O. Harris
> > >>>>>>>>>> Cloud Software Engineer
> > >>>>>>>>>> Sungard Availability Services
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Karl O. Harris
> > >>>>> Cloud Software Engineer
> > >>>>> Sungard Availability Services
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Karl O. Harris
> > >>> Cloud Software Engineer
> > >>> Sungard Availability Services
> > >>
> > >>
> > >>
> > >
> > >
> > > --
> > > Karl O. Harris
> > > Cloud Software Engineer
> > > Sungard Availability Services
> >
> >
>
>
> --
> Karl O. Harris
> Cloud Software Engineer
> Sungard Availability Services
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message