cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Harris <karl.har...@sungard.com>
Subject Re: Working on CloudStack Jira-764:nTier Apps 2.0 : Redundant Virtual Router for VPC email 2 of 2
Date Wed, 12 Feb 2014 19:15:56 GMT
On Wed, Feb 12, 2014 at 9:38 AM, Karl Harris <karl.harris@sungard.com>wrote:

> Would implementing a VPC Redundant Router Network Guru using the an
> extension of the NetworkGuru interface be an appropriate way to create a
> VPC network with redundant routers. It seems that extending Network Guru
> interface or extending one of it's children eg the abstract class
> GuestNetworkGuru would be a appropriate design path.
>
> Comments?
>
> Karl
>
I know the configuration of the VM which implement the redundant routers
are handled via BASH scripts and the environment variables int the VM
images are used to configure the conntrackd and keepalived applications via
SED in bash scripts. I am looking for a clean way to pass the variable
number of NIC's and then generate the LINK and IGNORE configuration
commands for conntrackd.

The Network Guru mentioned above seemed like an interesting place to look,
even though it's used to work with the DAO's for the NetworkElements.

Karl


>
> On Mon, Feb 3, 2014 at 2:58 PM, Daan Hoogland <daan.hoogland@gmail.com>wrote:
>
>> On Mon, Feb 3, 2014 at 8:50 PM, Sheng Yang <sheng@yasker.org> wrote:
>> > On Mon, Feb 3, 2014 at 11:27 AM, Karl Harris <karl.harris@sungard.com
>> >wrote:
>> >
>> >> On Mon, Feb 3, 2014 at 2:03 PM, Daan Hoogland <daan.hoogland@gmail.com
>> >> >wrote:
>> >>
>> >> >
>> >> > On 03 Feb 2014, at 19:45, Karl Harris <karl.harris@sungard.com>
>> wrote:
>> >> >
>> >> > > On Mon, Feb 3, 2014 at 12:54 PM, Daan Hoogland <
>> >> daan.hoogland@gmail.com
>> >> > >wrote:
>> >> > >
>> >> > >>
>> >> > >> On 03 Feb 2014, at 18:03, Karl Harris <karl.harris@sungard.com>
>> >> wrote:
>> >> > >>
>> >> > >>> After discussion with my colleagues  questions about initial
>> >> > >>> configuration of the open network redundant routers and
the
>> >> > >>> applicability of the existing bash scripts (cloud-early-config)
>> to
>> >> the
>> >> > >>> setup of VPC redundant routers have generated.
>> >> > >>>
>> >> > >>>
>> >> > >>> Some setup first: In the bash script cloud-early-config
there is
>> a
>> >> > >>> function named setup_redundant_router which makes copies
of
>> several
>> >> > >>> template files. The template files are used to configure
>> keepalived
>> >> > >>> and conntrackd. The template copies are edited, via sed
editor,
>> using
>> >> > >>> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.)
 which
>> are
>> >> > >>> obtained from the kernel of the current running linux
image
>> using the
>> >> > >>> virtual file system /proc/cmdline.
>> >> > >>>
>> >> > >>> I'm sure keepalived and conntrackd  can be used for starting
and
>> >> > >>> control of VPC redundant routers. However the setup of
>> keepalived and
>> >> > >>> conntrackd for VPC needs setup parameters which are dynamic
>> because a
>> >> > >>> VPC can have N number of redundant router pairs, not just
the
>> fixed
>> >> > >>> number parsed from proc/cmdline in the running kernel.
>> >> > >>>
>> >> > >>> Am I correct in this analysis?
>> >> > >> Karl, I think not. There is only one router in a vac, it routes
>> for
>> >> all
>> >> > >> networks in the virtual private cloud. Am I misreading your
>> >> description.
>> >> > >
>> >> > > Ok then a VPC should contain a single router containing N+1 (N+2
if
>> >> > using a
>> >> > > VPN connection) nic's? Where N is the number of private networks.
>> >> > And I would like to see N+3 as it will allocate another when a
>> private
>> >> > gateway is defined. That is why I proposed to pre-allocate this nic
>> to be
>> >> > able to predict mic-ids, i.e. eth<#>.
>> >> >
>> >> > >
>> >> > > If so then my original question still holds in the sense that
the
>> nic's
>> >> > > need to be created in the VM kernel based on the number of private
>> >> > networks
>> >> > > desired?
>> >> >
>> >> > Yes
>> >> >
>> >> > > I found the method update_System_Vm_Templates in the jar file
>> >> > > Upgrade410to420 in cloud-engine-schema/src/com/cloud/upgrade/dao
>> which
>> >> > > seems to point to a set of pre-configured VM images.
>> >> > I can not find the method you describe but these preconfigured
>> images are
>> >> > the ones used for all the system vms. That would be secondary storage
>> >> vms,
>> >> > console proxy vms and indeed the router vms.
>> >> >
>> >> > Based on the discussion above the the configuration part of the VPC
>> >> redundant router is:
>> >>
>> >>  1. Collecting the correct data required to configure the nics. (Is
>> this
>> >> part already in the VPC setup code? I will look for it, pointers will
>> be
>> >> helpful.)
>> >>
>> >
>> > vpc_guestgw.sh
>> and VpcVirtualNetworkApplianceManager and -Impl and their ancestors
>>
>>
>> >
>> >
>> >>  2. Based on the hypervisor/vm types configure the nic(s) on the
>> router VM
>> >> using the data collected above (Same question as in 1 above is this
>> code
>> >> already in place?). My guess is to start with a preconfigured template
>> and
>> >
>> > configure the vm based on the number of nics and any other appropriate
>> data.
>> >>
>> >
>> > Yes.
>> >
>> >
>> >>  3. Configuring conntrackd and keepalived after the router vm (nics)
>> have
>> >> been configured and the router(s) started or restarted? I still need to
>> >> work through this.
>> >>
>> >
>> > Likely you would need this.
>> >
>> > Don't know if keepalived can be start if there is no virtual IP
>> definition.
>> >
>> > You also need to think about what to do second router(or the router
>> pair)
>> > when the last tier of VPC has been destroyed.
>> >
>> > --Sheng
>> >
>> >
>> >>
>> >>
>> >>
>> >>
>> >> >
>> >> > >>
>> >> > >>> If so, given the dynamic nature of the VPC redundant router
>> >> > >>> configurations: Is using a setup_VPC_redundant_router
bash
>> function,
>> >> > >>> similar to the existing open network function mentioned
above,
>> the
>> >> > >>> most appropriate way to setup the keepalived or conntrackd
>> >> > >>> configuration files for VPC redundant routers in the
>> >> > >>> cloud-early-script? It seems to me reading the parameters
from
>> the
>> >> > >>> kernel will require a unwieldy set of kernels to match
the N
>> private
>> >> > >>> network redundant router pairs configured by the enduser.
>> >> > >>>
>> >> > >>>
>> >> > >>> Comments, questions, clarifications?
>> >> > >>
>> >> > >>>
>> >> > >>> Karl
>> >> > >>>
>> >> > >>>
>> >> > >>> In the bash script ea
>> >> > >>> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland <
>> >> > daan.hoogland@gmail.com>
>> >> > >> wrote:
>> >> > >>>> good reason to skip it for a next version, let's look
into it
>> >> anyway,
>> >> > >>>> as we don't want to burn any of our ships.
>> >> > >>>>
>> >> > >>>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris <
>> >> karl.harris@sungard.com
>> >> > >
>> >> > >> wrote:
>> >> > >>>>> All,
>> >> > >>>>>
>> >> > >>>>> At first redundant DHCP seemed like a good idea.
I did some
>> cursory
>> >> > >>>>> research and the more I read the more I'm convinced
it may be
>> >> > >>>>> more trouble than its worth for the first implementation.
I'll
>> talk
>> >> > >>>>> with some of our Systems Engineer's here and get
a broader
>> >> > >>>>> perspective.
>> >> > >>>>>
>> >> > >>>>> There seems to be only a single implementation
of an open
>> source
>> >> DHCP
>> >> > >>>>> server that will handle the synchronization required
for
>> redundant
>> >> > >>>>> servers.
>> >> > >>>>>
>> >> > >>>>> Karl
>> >> > >>>>>
>> >> > >>>>>
>> >> > >>>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland
<
>> >> > >> daan.hoogland@gmail.com> wrote:
>> >> > >>>>>> Saurav,
>> >> > >>>>>>
>> >> > >>>>>> Not sure how this happens now, but it is definateluy
>> something we
>> >> > >>>>>> want. For the static conf files it won't be
much of a
>> problem. The
>> >> > >>>>>> firewall/loadbalences won't be much of a problem,
they are
>> fire
>> >> and
>> >> > >>>>>> forget configurations of the ms that can just
be sent to
>> both. The
>> >> > >>>>>> dhcp is a challange. I am not sure if it is
solved for the
>> plain
>> >> rvr
>> >> > >>>>>> now but it should be solved for that as well.
>> >> > >>>>>>
>> >> > >>>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri
>> >> > >>>>>> <saurav.lahiri@sungard.com> wrote:
>> >> > >>>>>>> Daan,
>> >> > >>>>>>> I was wondering if you had not shared
your thoughts, but
>> looks
>> >> like
>> >> > >> I had
>> >> > >>>>>>> missed your mail.
>> >> > >>>>>>>
>> >> > >>>>>>> I agree that redundant dhcp or dns would
be good to have.
>> What I
>> >> > >> meant was,
>> >> > >>>>>>> it appears that by  just enabling RVR
  there is no way to
>> auto
>> >> > sync
>> >> > >>>>>>> configuration between the  master and
slave nodes with
>> regard to
>> >> > >> dhcp,
>> >> > >>>>>>> loadbalancer and firewall(specifically
the dhcp lease file,
>> >> > >> haproxy,cfg and
>> >> > >>>>>>> iptables configuration).  So just enabling
RVR does not
>> ensure
>> >> high
>> >> > >>>>>>> availability for  these services. Is there
a way cloudstack
>> >> > autosyncs
>> >> > >>>>>>> configuration?
>> >> > >>>>>>>
>> >> > >>>>>>> For the routing portion this is not an
issue as the
>> participating
>> >> > >> routers
>> >> > >>>>>>> learn the route through known protocols.
>> >> > >>>>>>>
>> >> > >>>>>>> Thanks
>> >> > >>>>>>> Saurav
>> >> > >>>>>>>
>> >> > >>>>>>>
>> >> > >>>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan
Hoogland <
>> >> > >> daan.hoogland@gmail.com>wrote:
>> >> > >>>>>>>
>> >> > >>>>>>>> Saurav, I don't see why you can't
benefit from having other
>> >> > services
>> >> > >>>>>>>> redundant as well. Vpn might be a
problem as the source ip
>> of a
>> >> > >>>>>>>> redundant router pair on a vpn might
give a problem (maybe
>> there
>> >> > is
>> >> > >> an
>> >> > >>>>>>>> implementation) but why wouldn't you
want redundant dhcp or
>> dns
>> >> > >>>>>>>> services? As I understand it these
are used at Schuberg
>> Philis
>> >> at
>> >> > >> the
>> >> > >>>>>>>> moment. will double check when I get
a chance.
>> >> > >>>>>>>>
>> >> > >>>>>>>> regards,
>> >> > >>>>>>>> Daan
>> >> > >>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav
Lahiri
>> >> > >>>>>>>> <saurav.lahiri@sungard.com>
wrote:
>> >> > >>>>>>>>> Daan,
>> >> > >>>>>>>>> So looking at what is available
today for guest network,
>> the
>> >> > >> Redundant VR
>> >> > >>>>>>>>> can be enabled only for the source
nat service. I guess the
>> >> > >> mention of
>> >> > >>>>>>>> the
>> >> > >>>>>>>>> source nat router is in relation
to that. I could be wrong
>> >> > though.
>> >> > >> It
>> >> > >>>>>>>>> appears  that the other services
like vpn, dhcp, dns do not
>> >> > >> benefit much
>> >> > >>>>>>>>> from the RVR capability. Can you
clarify if thats correct?
>> >> > >>>>>>>>>
>> >> > >>>>>>>>> Thanks
>> >> > >>>>>>>>> Saurav
>> >> > >>>>>>>>>
>> >> > >>>>>>>>>
>> >> > >>>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM,
Karl Harris <
>> >> > >> karl.harris@sungard.com
>> >> > >>>>>>>>> wrote:
>> >> > >>>>>>>>>
>> >> > >>>>>>>>>> ---------- Forwarded message
----------
>> >> > >>>>>>>>>> From: Daan Hoogland <daan.hoogland@gmail.com>
>> >> > >>>>>>>>>> Date: Wed, Jan 15, 2014 at
2:51 PM
>> >> > >>>>>>>>>> Subject: Re: rvr4vpc
>> >> > >>>>>>>>>> To: Karl Harris <karl.harris@sungard.com>
>> >> > >>>>>>>>>> Cc: Christopher Litsinger
<
>> christopher.litsinge@sungard.com>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> H Karl,
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> Thanks for sharing. I didn't
want to initiate this but I
>> >> > >> encourage you
>> >> > >>>>>>>>>> to share this on the dev list
(not in jira) as things are
>> only
>> >> > >>>>>>>>>> considered 'discussed' if
they passed by there.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> You speak of '1 Get configuration
data on Source Nat
>> Router',
>> >> I
>> >> > >> don't
>> >> > >>>>>>>>>> understand why you call the
router by this designation.
>> >> 'Source
>> >> > >> Nat'
>> >> > >>>>>>>>>> is only one of it's many possible
functions.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> Apart from the design principles
I shared with you I have
>> so
>> >> far
>> >> > >> only
>> >> > >>>>>>>>>> a technical implementation
detail so far. That is to
>> reserve
>> >> the
>> >> > >>>>>>>>>> (eth2) interface for the private
gateway on the vpc (r)vr.
>> >> This
>> >> > >> way
>> >> > >>>>>>>>>> the inteface to configure
are somewhat predictable.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> As for the design principle
to have a statefull router
>> (reboot
>> >> > >> proof)
>> >> > >>>>>>>>>> the idea is to implement a
configuration file that will be
>> >> > >> uploaded to
>> >> > >>>>>>>>>> the router after which a self-config
command is send that
>> will
>> >> > >>>>>>>>>> implement the details of configuring
the interfaces,
>> haproxy
>> >> and
>> >> > >>>>>>>>>> keepalived and maybe more.
I think your current
>> assessment of
>> >> > the
>> >> > >>>>>>>>>> working of the RVRs is accurate
but it will not be
>> workable
>> >> for
>> >> > an
>> >> > >>>>>>>>>> implementation for vpc's as
they have an unpreditable
>> number
>> >> of
>> >> > >>>>>>>>>> interfaces.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> to bad you can't make it next
thursday,
>> >> > >>>>>>>>>> Daan Hoogland
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> On Wed, Jan 15, 2014 at 3:25
PM, Karl Harris <
>> >> > >> karl.harris@sungard.com>
>> >> > >>>>>>>>>> wrote:
>> >> > >>>>>>>>>>> Daan,
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Sorry for the delayed
response but as Christopher
>> mentioned
>> >> to
>> >> > >> you in
>> >> > >>>>>>>> his
>> >> > >>>>>>>>>>> email I am getting my
head around the CloudStack
>> software.
>> >> > >>>>>>>>>>> Since I am new to CloudStack
but "old" to enterprise
>> level
>> >> JAVA
>> >> > >> the
>> >> > >>>>>>>> task
>> >> > >>>>>>>>>> is
>> >> > >>>>>>>>>>> large but not impossible.
I have no experience with
>> running
>> >> > >> CloudStack
>> >> > >>>>>>>>>> but
>> >> > >>>>>>>>>>> considerable experience
designing and maintaining large
>> JAVA
>> >> > >>>>>>>>>> applications.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> I've created what I believe
is a very high level
>> abstract of
>> >> > how
>> >> > >> the
>> >> > >>>>>>>>>> current
>> >> > >>>>>>>>>>> guest VRR's are created
for guest networks with the
>> intent of
>> >> > >> making
>> >> > >>>>>>>> this
>> >> > >>>>>>>>>>> abstract
>> >> > >>>>>>>>>>> more detailed.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> 1 Get configuration data
on Source Nat Router  selected
>> as a
>> >> > >> redundant
>> >> > >>>>>>>>>>> router
>> >> > >>>>>>>>>>>  1.1 Public and Guest
network identified.
>> >> > >>>>>>>>>>> 2 Both routers are provisioned
>> >> > >>>>>>>>>>>  2.1 Software  trys different,
>> >> > >> regions(?),zones,pods,clusters,hosts
>> >> > >>>>>>>> in
>> >> > >>>>>>>>>>> that order as the location
of the router. Log maximum
>> >> > "distance"
>> >> > >>>>>>>> apart.
>> >> > >>>>>>>>>>> 3 Keepalived is configured
>> >> > >>>>>>>>>>> 4 Both routers are started
>> >> > >>>>>>>>>>> 5 Keepalived is started
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Obviously there is much
more that is happening under
>> each of
>> >> > the
>> >> > >> steps
>> >> > >>>>>>>>>>> above. My intent is to
complete this detailed "as is"
>> listing
>> >> > as
>> >> > >> much
>> >> > >>>>>>>> as
>> >> > >>>>>>>>>> we
>> >> > >>>>>>>>>>> can. Then  using the "as
is" description/sequence
>> >> > >>>>>>>>>>> make a "to-be" addition
for VPC's. When I get a
>> consensus on
>> >> > WHAT
>> >> > >>>>>>>> needs
>> >> > >>>>>>>>>> to
>> >> > >>>>>>>>>>> be implemented for the
VRR in VPC  then develop HOW best
>> to
>> >> > >> implement
>> >> > >>>>>>>> the
>> >> > >>>>>>>>>>> "to-be" addition with
the
>> >> > >>>>>>>>>>> existing JAVA code as
well as what additional classes
>> need to
>> >> > be
>> >> > >>>>>>>>>>> extended/implemented/created.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Comments, critiques and
changes to the above sequence are
>> >> > >> encouraged.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> I plan on posting this
to the dev-list/Jira very soon.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> I have been using this
functional spec as a guide, after
>> >> > >> discussing
>> >> > >>>>>>>> this
>> >> > >>>>>>>>>>> with our Systems Engineering
people this spec meets our
>> >> > >> requirements.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Do you have an implementation
in mind?
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> We have an internal Cloud
Meeting with conflicts with the
>> >> > >> cloudstack
>> >> > >>>>>>>>>> "day"
>> >> > >>>>>>>>>>> next week so I will not
be in attendence.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Karl
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> On Tue, Jan 14, 2014 at
4:35 PM, Daan Hoogland <
>> >> > >>>>>>>> daan.hoogland@gmail.com>
>> >> > >>>>>>>>>>> wrote:
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> Hello overthere in
the states,
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> Tomorow I will start
some experimenting with redundant
>> vpc
>> >> > >> routers.
>> >> > >>>>>>>>>>>> This is to check up
on any findings and requirements
>> that
>> >> you
>> >> > >> might
>> >> > >>>>>>>>>>>> have on this. Once
again I would not like to waste work
>> on
>> >> > this
>> >> > >> as it
>> >> > >>>>>>>>>>>> is really a globally
usable feature that is probably
>> >> > universal.
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> please let me know
your status on this.
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> If any of you are
coming to the cloudstack day in London
>> >> next
>> >> > >> week,
>> >> > >>>>>>>>>>>> let's meetup next
thursday.
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> kind regards,
>> >> > >>>>>>>>>>>> Daan Hoogland
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> --
>> >> > >>>>>>>>>>> Karl O. Harris
>> >> > >>>>>>>>>>> Cloud Software Engineer
>> >> > >>>>>>>>>>> Sungard Availability Services
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> --
>> >> > >>>>>>>>>> Karl O. Harris
>> >> > >>>>>>>>>> Cloud Software Engineer
>> >> > >>>>>>>>>> Sungard Availability Services
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>
>> >> > >>>>>
>> >> > >>>>>
>> >> > >>>>>
>> >> > >>>>> --
>> >> > >>>>> Karl O. Harris
>> >> > >>>>> Cloud Software Engineer
>> >> > >>>>> Sungard Availability Services
>> >> > >>>>
>> >> > >>>
>> >> > >>>
>> >> > >>>
>> >> > >>> --
>> >> > >>> Karl O. Harris
>> >> > >>> Cloud Software Engineer
>> >> > >>> Sungard Availability Services
>> >> > >>
>> >> > >>
>> >> > >>
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Karl O. Harris
>> >> > > Cloud Software Engineer
>> >> > > Sungard Availability Services
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Karl O. Harris
>> >> Cloud Software Engineer
>> >> Sungard Availability Services
>> >>
>>
>>
>>
>> --
>> Daan
>>
>>
>
>
> --
> Karl O. Harris
> Cloud Software Engineer
> Sungard Availability Services
>
>


-- 
Karl O. Harris
Cloud Software Engineer
Sungard Availability Services

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message