cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Harris <karl.har...@sungard.com>
Subject Re: Working on CloudStack Jira-764:nTier Apps 2.0 : Redundant Virtual Router for VPC email 2 of 2
Date Wed, 12 Feb 2014 14:38:18 GMT
Would implementing a VPC Redundant Router Network Guru using the an
extension of the NetworkGuru interface be an appropriate way to create a
VPC network with redundant routers. It seems that extending Network Guru
interface or extending one of it's children eg the abstract class
GuestNetworkGuru would be a appropriate design path.

Comments?

Karl



On Mon, Feb 3, 2014 at 2:58 PM, Daan Hoogland <daan.hoogland@gmail.com>wrote:

> On Mon, Feb 3, 2014 at 8:50 PM, Sheng Yang <sheng@yasker.org> wrote:
> > On Mon, Feb 3, 2014 at 11:27 AM, Karl Harris <karl.harris@sungard.com
> >wrote:
> >
> >> On Mon, Feb 3, 2014 at 2:03 PM, Daan Hoogland <daan.hoogland@gmail.com
> >> >wrote:
> >>
> >> >
> >> > On 03 Feb 2014, at 19:45, Karl Harris <karl.harris@sungard.com>
> wrote:
> >> >
> >> > > On Mon, Feb 3, 2014 at 12:54 PM, Daan Hoogland <
> >> daan.hoogland@gmail.com
> >> > >wrote:
> >> > >
> >> > >>
> >> > >> On 03 Feb 2014, at 18:03, Karl Harris <karl.harris@sungard.com>
> >> wrote:
> >> > >>
> >> > >>> After discussion with my colleagues  questions about initial
> >> > >>> configuration of the open network redundant routers and the
> >> > >>> applicability of the existing bash scripts (cloud-early-config)
to
> >> the
> >> > >>> setup of VPC redundant routers have generated.
> >> > >>>
> >> > >>>
> >> > >>> Some setup first: In the bash script cloud-early-config there
is a
> >> > >>> function named setup_redundant_router which makes copies of
> several
> >> > >>> template files. The template files are used to configure
> keepalived
> >> > >>> and conntrackd. The template copies are edited, via sed editor,
> using
> >> > >>> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.) 
which
> are
> >> > >>> obtained from the kernel of the current running linux image
using
> the
> >> > >>> virtual file system /proc/cmdline.
> >> > >>>
> >> > >>> I'm sure keepalived and conntrackd  can be used for starting
and
> >> > >>> control of VPC redundant routers. However the setup of keepalived
> and
> >> > >>> conntrackd for VPC needs setup parameters which are dynamic
> because a
> >> > >>> VPC can have N number of redundant router pairs, not just
the
> fixed
> >> > >>> number parsed from proc/cmdline in the running kernel.
> >> > >>>
> >> > >>> Am I correct in this analysis?
> >> > >> Karl, I think not. There is only one router in a vac, it routes
for
> >> all
> >> > >> networks in the virtual private cloud. Am I misreading your
> >> description.
> >> > >
> >> > > Ok then a VPC should contain a single router containing N+1 (N+2 if
> >> > using a
> >> > > VPN connection) nic's? Where N is the number of private networks.
> >> > And I would like to see N+3 as it will allocate another when a private
> >> > gateway is defined. That is why I proposed to pre-allocate this nic
> to be
> >> > able to predict mic-ids, i.e. eth<#>.
> >> >
> >> > >
> >> > > If so then my original question still holds in the sense that the
> nic's
> >> > > need to be created in the VM kernel based on the number of private
> >> > networks
> >> > > desired?
> >> >
> >> > Yes
> >> >
> >> > > I found the method update_System_Vm_Templates in the jar file
> >> > > Upgrade410to420 in cloud-engine-schema/src/com/cloud/upgrade/dao
> which
> >> > > seems to point to a set of pre-configured VM images.
> >> > I can not find the method you describe but these preconfigured images
> are
> >> > the ones used for all the system vms. That would be secondary storage
> >> vms,
> >> > console proxy vms and indeed the router vms.
> >> >
> >> > Based on the discussion above the the configuration part of the VPC
> >> redundant router is:
> >>
> >>  1. Collecting the correct data required to configure the nics. (Is this
> >> part already in the VPC setup code? I will look for it, pointers will be
> >> helpful.)
> >>
> >
> > vpc_guestgw.sh
> and VpcVirtualNetworkApplianceManager and -Impl and their ancestors
>
>
> >
> >
> >>  2. Based on the hypervisor/vm types configure the nic(s) on the router
> VM
> >> using the data collected above (Same question as in 1 above is this code
> >> already in place?). My guess is to start with a preconfigured template
> and
> >
> > configure the vm based on the number of nics and any other appropriate
> data.
> >>
> >
> > Yes.
> >
> >
> >>  3. Configuring conntrackd and keepalived after the router vm (nics)
> have
> >> been configured and the router(s) started or restarted? I still need to
> >> work through this.
> >>
> >
> > Likely you would need this.
> >
> > Don't know if keepalived can be start if there is no virtual IP
> definition.
> >
> > You also need to think about what to do second router(or the router pair)
> > when the last tier of VPC has been destroyed.
> >
> > --Sheng
> >
> >
> >>
> >>
> >>
> >>
> >> >
> >> > >>
> >> > >>> If so, given the dynamic nature of the VPC redundant router
> >> > >>> configurations: Is using a setup_VPC_redundant_router bash
> function,
> >> > >>> similar to the existing open network function mentioned above,
the
> >> > >>> most appropriate way to setup the keepalived or conntrackd
> >> > >>> configuration files for VPC redundant routers in the
> >> > >>> cloud-early-script? It seems to me reading the parameters
from the
> >> > >>> kernel will require a unwieldy set of kernels to match the
N
> private
> >> > >>> network redundant router pairs configured by the enduser.
> >> > >>>
> >> > >>>
> >> > >>> Comments, questions, clarifications?
> >> > >>
> >> > >>>
> >> > >>> Karl
> >> > >>>
> >> > >>>
> >> > >>> In the bash script ea
> >> > >>> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland <
> >> > daan.hoogland@gmail.com>
> >> > >> wrote:
> >> > >>>> good reason to skip it for a next version, let's look
into it
> >> anyway,
> >> > >>>> as we don't want to burn any of our ships.
> >> > >>>>
> >> > >>>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris <
> >> karl.harris@sungard.com
> >> > >
> >> > >> wrote:
> >> > >>>>> All,
> >> > >>>>>
> >> > >>>>> At first redundant DHCP seemed like a good idea. I
did some
> cursory
> >> > >>>>> research and the more I read the more I'm convinced
it may be
> >> > >>>>> more trouble than its worth for the first implementation.
I'll
> talk
> >> > >>>>> with some of our Systems Engineer's here and get a
broader
> >> > >>>>> perspective.
> >> > >>>>>
> >> > >>>>> There seems to be only a single implementation of
an open source
> >> DHCP
> >> > >>>>> server that will handle the synchronization required
for
> redundant
> >> > >>>>> servers.
> >> > >>>>>
> >> > >>>>> Karl
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland <
> >> > >> daan.hoogland@gmail.com> wrote:
> >> > >>>>>> Saurav,
> >> > >>>>>>
> >> > >>>>>> Not sure how this happens now, but it is definateluy
something
> we
> >> > >>>>>> want. For the static conf files it won't be much
of a problem.
> The
> >> > >>>>>> firewall/loadbalences won't be much of a problem,
they are fire
> >> and
> >> > >>>>>> forget configurations of the ms that can just
be sent to both.
> The
> >> > >>>>>> dhcp is a challange. I am not sure if it is solved
for the
> plain
> >> rvr
> >> > >>>>>> now but it should be solved for that as well.
> >> > >>>>>>
> >> > >>>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri
> >> > >>>>>> <saurav.lahiri@sungard.com> wrote:
> >> > >>>>>>> Daan,
> >> > >>>>>>> I was wondering if you had not shared your
thoughts, but looks
> >> like
> >> > >> I had
> >> > >>>>>>> missed your mail.
> >> > >>>>>>>
> >> > >>>>>>> I agree that redundant dhcp or dns would be
good to have.
> What I
> >> > >> meant was,
> >> > >>>>>>> it appears that by  just enabling RVR   there
is no way to
> auto
> >> > sync
> >> > >>>>>>> configuration between the  master and slave
nodes with regard
> to
> >> > >> dhcp,
> >> > >>>>>>> loadbalancer and firewall(specifically the
dhcp lease file,
> >> > >> haproxy,cfg and
> >> > >>>>>>> iptables configuration).  So just enabling
RVR does not ensure
> >> high
> >> > >>>>>>> availability for  these services. Is there
a way cloudstack
> >> > autosyncs
> >> > >>>>>>> configuration?
> >> > >>>>>>>
> >> > >>>>>>> For the routing portion this is not an issue
as the
> participating
> >> > >> routers
> >> > >>>>>>> learn the route through known protocols.
> >> > >>>>>>>
> >> > >>>>>>> Thanks
> >> > >>>>>>> Saurav
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan Hoogland
<
> >> > >> daan.hoogland@gmail.com>wrote:
> >> > >>>>>>>
> >> > >>>>>>>> Saurav, I don't see why you can't benefit
from having other
> >> > services
> >> > >>>>>>>> redundant as well. Vpn might be a problem
as the source ip
> of a
> >> > >>>>>>>> redundant router pair on a vpn might give
a problem (maybe
> there
> >> > is
> >> > >> an
> >> > >>>>>>>> implementation) but why wouldn't you want
redundant dhcp or
> dns
> >> > >>>>>>>> services? As I understand it these are
used at Schuberg
> Philis
> >> at
> >> > >> the
> >> > >>>>>>>> moment. will double check when I get a
chance.
> >> > >>>>>>>>
> >> > >>>>>>>> regards,
> >> > >>>>>>>> Daan
> >> > >>>>>>>>
> >> > >>>>>>>>
> >> > >>>>>>>>
> >> > >>>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav
Lahiri
> >> > >>>>>>>> <saurav.lahiri@sungard.com> wrote:
> >> > >>>>>>>>> Daan,
> >> > >>>>>>>>> So looking at what is available today
for guest network, the
> >> > >> Redundant VR
> >> > >>>>>>>>> can be enabled only for the source
nat service. I guess the
> >> > >> mention of
> >> > >>>>>>>> the
> >> > >>>>>>>>> source nat router is in relation to
that. I could be wrong
> >> > though.
> >> > >> It
> >> > >>>>>>>>> appears  that the other services like
vpn, dhcp, dns do not
> >> > >> benefit much
> >> > >>>>>>>>> from the RVR capability. Can you clarify
if thats correct?
> >> > >>>>>>>>>
> >> > >>>>>>>>> Thanks
> >> > >>>>>>>>> Saurav
> >> > >>>>>>>>>
> >> > >>>>>>>>>
> >> > >>>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM, Karl
Harris <
> >> > >> karl.harris@sungard.com
> >> > >>>>>>>>> wrote:
> >> > >>>>>>>>>
> >> > >>>>>>>>>> ---------- Forwarded message ----------
> >> > >>>>>>>>>> From: Daan Hoogland <daan.hoogland@gmail.com>
> >> > >>>>>>>>>> Date: Wed, Jan 15, 2014 at 2:51
PM
> >> > >>>>>>>>>> Subject: Re: rvr4vpc
> >> > >>>>>>>>>> To: Karl Harris <karl.harris@sungard.com>
> >> > >>>>>>>>>> Cc: Christopher Litsinger <
> christopher.litsinge@sungard.com>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> H Karl,
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> Thanks for sharing. I didn't want
to initiate this but I
> >> > >> encourage you
> >> > >>>>>>>>>> to share this on the dev list
(not in jira) as things are
> only
> >> > >>>>>>>>>> considered 'discussed' if they
passed by there.
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> You speak of '1 Get configuration
data on Source Nat
> Router',
> >> I
> >> > >> don't
> >> > >>>>>>>>>> understand why you call the router
by this designation.
> >> 'Source
> >> > >> Nat'
> >> > >>>>>>>>>> is only one of it's many possible
functions.
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> Apart from the design principles
I shared with you I have
> so
> >> far
> >> > >> only
> >> > >>>>>>>>>> a technical implementation detail
so far. That is to
> reserve
> >> the
> >> > >>>>>>>>>> (eth2) interface for the private
gateway on the vpc (r)vr.
> >> This
> >> > >> way
> >> > >>>>>>>>>> the inteface to configure are
somewhat predictable.
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> As for the design principle to
have a statefull router
> (reboot
> >> > >> proof)
> >> > >>>>>>>>>> the idea is to implement a configuration
file that will be
> >> > >> uploaded to
> >> > >>>>>>>>>> the router after which a self-config
command is send that
> will
> >> > >>>>>>>>>> implement the details of configuring
the interfaces,
> haproxy
> >> and
> >> > >>>>>>>>>> keepalived and maybe more. I think
your current assessment
> of
> >> > the
> >> > >>>>>>>>>> working of the RVRs is accurate
but it will not be workable
> >> for
> >> > an
> >> > >>>>>>>>>> implementation for vpc's as they
have an unpreditable
> number
> >> of
> >> > >>>>>>>>>> interfaces.
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> to bad you can't make it next
thursday,
> >> > >>>>>>>>>> Daan Hoogland
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> On Wed, Jan 15, 2014 at 3:25 PM,
Karl Harris <
> >> > >> karl.harris@sungard.com>
> >> > >>>>>>>>>> wrote:
> >> > >>>>>>>>>>> Daan,
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> Sorry for the delayed response
but as Christopher
> mentioned
> >> to
> >> > >> you in
> >> > >>>>>>>> his
> >> > >>>>>>>>>>> email I am getting my head
around the CloudStack software.
> >> > >>>>>>>>>>> Since I am new to CloudStack
but "old" to enterprise level
> >> JAVA
> >> > >> the
> >> > >>>>>>>> task
> >> > >>>>>>>>>> is
> >> > >>>>>>>>>>> large but not impossible.
I have no experience with
> running
> >> > >> CloudStack
> >> > >>>>>>>>>> but
> >> > >>>>>>>>>>> considerable experience designing
and maintaining large
> JAVA
> >> > >>>>>>>>>> applications.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> I've created what I believe
is a very high level abstract
> of
> >> > how
> >> > >> the
> >> > >>>>>>>>>> current
> >> > >>>>>>>>>>> guest VRR's are created for
guest networks with the
> intent of
> >> > >> making
> >> > >>>>>>>> this
> >> > >>>>>>>>>>> abstract
> >> > >>>>>>>>>>> more detailed.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> 1 Get configuration data on
Source Nat Router  selected
> as a
> >> > >> redundant
> >> > >>>>>>>>>>> router
> >> > >>>>>>>>>>>  1.1 Public and Guest network
identified.
> >> > >>>>>>>>>>> 2 Both routers are provisioned
> >> > >>>>>>>>>>>  2.1 Software  trys different,
> >> > >> regions(?),zones,pods,clusters,hosts
> >> > >>>>>>>> in
> >> > >>>>>>>>>>> that order as the location
of the router. Log maximum
> >> > "distance"
> >> > >>>>>>>> apart.
> >> > >>>>>>>>>>> 3 Keepalived is configured
> >> > >>>>>>>>>>> 4 Both routers are started
> >> > >>>>>>>>>>> 5 Keepalived is started
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> Obviously there is much more
that is happening under each
> of
> >> > the
> >> > >> steps
> >> > >>>>>>>>>>> above. My intent is to complete
this detailed "as is"
> listing
> >> > as
> >> > >> much
> >> > >>>>>>>> as
> >> > >>>>>>>>>> we
> >> > >>>>>>>>>>> can. Then  using the "as is"
description/sequence
> >> > >>>>>>>>>>> make a "to-be" addition for
VPC's. When I get a consensus
> on
> >> > WHAT
> >> > >>>>>>>> needs
> >> > >>>>>>>>>> to
> >> > >>>>>>>>>>> be implemented for the VRR
in VPC  then develop HOW best
> to
> >> > >> implement
> >> > >>>>>>>> the
> >> > >>>>>>>>>>> "to-be" addition with the
> >> > >>>>>>>>>>> existing JAVA code as well
as what additional classes
> need to
> >> > be
> >> > >>>>>>>>>>> extended/implemented/created.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> Comments, critiques and changes
to the above sequence are
> >> > >> encouraged.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> I plan on posting this to
the dev-list/Jira very soon.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> I have been using this functional
spec as a guide, after
> >> > >> discussing
> >> > >>>>>>>> this
> >> > >>>>>>>>>>> with our Systems Engineering
people this spec meets our
> >> > >> requirements.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> Do you have an implementation
in mind?
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> We have an internal Cloud
Meeting with conflicts with the
> >> > >> cloudstack
> >> > >>>>>>>>>> "day"
> >> > >>>>>>>>>>> next week so I will not be
in attendence.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> Karl
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> On Tue, Jan 14, 2014 at 4:35
PM, Daan Hoogland <
> >> > >>>>>>>> daan.hoogland@gmail.com>
> >> > >>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> Hello overthere in the
states,
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> Tomorow I will start some
experimenting with redundant
> vpc
> >> > >> routers.
> >> > >>>>>>>>>>>> This is to check up on
any findings and requirements that
> >> you
> >> > >> might
> >> > >>>>>>>>>>>> have on this. Once again
I would not like to waste work
> on
> >> > this
> >> > >> as it
> >> > >>>>>>>>>>>> is really a globally usable
feature that is probably
> >> > universal.
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> please let me know your
status on this.
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> If any of you are coming
to the cloudstack day in London
> >> next
> >> > >> week,
> >> > >>>>>>>>>>>> let's meetup next thursday.
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> kind regards,
> >> > >>>>>>>>>>>> Daan Hoogland
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> --
> >> > >>>>>>>>>>> Karl O. Harris
> >> > >>>>>>>>>>> Cloud Software Engineer
> >> > >>>>>>>>>>> Sungard Availability Services
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>> --
> >> > >>>>>>>>>> Karl O. Harris
> >> > >>>>>>>>>> Cloud Software Engineer
> >> > >>>>>>>>>> Sungard Availability Services
> >> > >>>>>>>>>>
> >> > >>>>>>>>
> >> > >>>>>>>>
> >> > >>>>>>
> >> > >>>>>
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> --
> >> > >>>>> Karl O. Harris
> >> > >>>>> Cloud Software Engineer
> >> > >>>>> Sungard Availability Services
> >> > >>>>
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> --
> >> > >>> Karl O. Harris
> >> > >>> Cloud Software Engineer
> >> > >>> Sungard Availability Services
> >> > >>
> >> > >>
> >> > >>
> >> > >
> >> > >
> >> > > --
> >> > > Karl O. Harris
> >> > > Cloud Software Engineer
> >> > > Sungard Availability Services
> >> >
> >> >
> >>
> >>
> >> --
> >> Karl O. Harris
> >> Cloud Software Engineer
> >> Sungard Availability Services
> >>
>
>
>
> --
> Daan
>
>


-- 
Karl O. Harris
Cloud Software Engineer
Sungard Availability Services

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message