cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daan Hoogland <daan.hoogl...@gmail.com>
Subject Re: Working on CloudStack Jira-764:nTier Apps 2.0 : Redundant Virtual Router for VPC email 2 of 2
Date Wed, 12 Feb 2014 19:43:04 GMT
H Karl,

I wouldn't say so. The network is redundant by instantiation and not
by design I would say. Maybe you are thinking of an other
implementation then you are. I haven't read any recent docs by you so
maybe I lack context. I would say that the network(s) are plain (sdn)
networks and the redundancy is implemented in the
VpcVirtualApplianceManager.

sorry I can't chip in at the moment. Hope you'll let me know when you
are up to partitioning the problem.

regards,



On Wed, Feb 12, 2014 at 3:38 PM, Karl Harris <karl.harris@sungard.com> wrote:
> Would implementing a VPC Redundant Router Network Guru using the an
> extension of the NetworkGuru interface be an appropriate way to create a
> VPC network with redundant routers. It seems that extending Network Guru
> interface or extending one of it's children eg the abstract class
> GuestNetworkGuru would be a appropriate design path.
>
> Comments?
>
> Karl
>
>
>
> On Mon, Feb 3, 2014 at 2:58 PM, Daan Hoogland <daan.hoogland@gmail.com>wrote:
>
>> On Mon, Feb 3, 2014 at 8:50 PM, Sheng Yang <sheng@yasker.org> wrote:
>> > On Mon, Feb 3, 2014 at 11:27 AM, Karl Harris <karl.harris@sungard.com
>> >wrote:
>> >
>> >> On Mon, Feb 3, 2014 at 2:03 PM, Daan Hoogland <daan.hoogland@gmail.com
>> >> >wrote:
>> >>
>> >> >
>> >> > On 03 Feb 2014, at 19:45, Karl Harris <karl.harris@sungard.com>
>> wrote:
>> >> >
>> >> > > On Mon, Feb 3, 2014 at 12:54 PM, Daan Hoogland <
>> >> daan.hoogland@gmail.com
>> >> > >wrote:
>> >> > >
>> >> > >>
>> >> > >> On 03 Feb 2014, at 18:03, Karl Harris <karl.harris@sungard.com>
>> >> wrote:
>> >> > >>
>> >> > >>> After discussion with my colleagues  questions about initial
>> >> > >>> configuration of the open network redundant routers and
the
>> >> > >>> applicability of the existing bash scripts (cloud-early-config)
to
>> >> the
>> >> > >>> setup of VPC redundant routers have generated.
>> >> > >>>
>> >> > >>>
>> >> > >>> Some setup first: In the bash script cloud-early-config
there is a
>> >> > >>> function named setup_redundant_router which makes copies
of
>> several
>> >> > >>> template files. The template files are used to configure
>> keepalived
>> >> > >>> and conntrackd. The template copies are edited, via sed
editor,
>> using
>> >> > >>> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.)
 which
>> are
>> >> > >>> obtained from the kernel of the current running linux
image using
>> the
>> >> > >>> virtual file system /proc/cmdline.
>> >> > >>>
>> >> > >>> I'm sure keepalived and conntrackd  can be used for starting
and
>> >> > >>> control of VPC redundant routers. However the setup of
keepalived
>> and
>> >> > >>> conntrackd for VPC needs setup parameters which are dynamic
>> because a
>> >> > >>> VPC can have N number of redundant router pairs, not just
the
>> fixed
>> >> > >>> number parsed from proc/cmdline in the running kernel.
>> >> > >>>
>> >> > >>> Am I correct in this analysis?
>> >> > >> Karl, I think not. There is only one router in a vac, it routes
for
>> >> all
>> >> > >> networks in the virtual private cloud. Am I misreading your
>> >> description.
>> >> > >
>> >> > > Ok then a VPC should contain a single router containing N+1 (N+2
if
>> >> > using a
>> >> > > VPN connection) nic's? Where N is the number of private networks.
>> >> > And I would like to see N+3 as it will allocate another when a private
>> >> > gateway is defined. That is why I proposed to pre-allocate this nic
>> to be
>> >> > able to predict mic-ids, i.e. eth<#>.
>> >> >
>> >> > >
>> >> > > If so then my original question still holds in the sense that
the
>> nic's
>> >> > > need to be created in the VM kernel based on the number of private
>> >> > networks
>> >> > > desired?
>> >> >
>> >> > Yes
>> >> >
>> >> > > I found the method update_System_Vm_Templates in the jar file
>> >> > > Upgrade410to420 in cloud-engine-schema/src/com/cloud/upgrade/dao
>> which
>> >> > > seems to point to a set of pre-configured VM images.
>> >> > I can not find the method you describe but these preconfigured images
>> are
>> >> > the ones used for all the system vms. That would be secondary storage
>> >> vms,
>> >> > console proxy vms and indeed the router vms.
>> >> >
>> >> > Based on the discussion above the the configuration part of the VPC
>> >> redundant router is:
>> >>
>> >>  1. Collecting the correct data required to configure the nics. (Is this
>> >> part already in the VPC setup code? I will look for it, pointers will be
>> >> helpful.)
>> >>
>> >
>> > vpc_guestgw.sh
>> and VpcVirtualNetworkApplianceManager and -Impl and their ancestors
>>
>>
>> >
>> >
>> >>  2. Based on the hypervisor/vm types configure the nic(s) on the router
>> VM
>> >> using the data collected above (Same question as in 1 above is this code
>> >> already in place?). My guess is to start with a preconfigured template
>> and
>> >
>> > configure the vm based on the number of nics and any other appropriate
>> data.
>> >>
>> >
>> > Yes.
>> >
>> >
>> >>  3. Configuring conntrackd and keepalived after the router vm (nics)
>> have
>> >> been configured and the router(s) started or restarted? I still need to
>> >> work through this.
>> >>
>> >
>> > Likely you would need this.
>> >
>> > Don't know if keepalived can be start if there is no virtual IP
>> definition.
>> >
>> > You also need to think about what to do second router(or the router pair)
>> > when the last tier of VPC has been destroyed.
>> >
>> > --Sheng
>> >
>> >
>> >>
>> >>
>> >>
>> >>
>> >> >
>> >> > >>
>> >> > >>> If so, given the dynamic nature of the VPC redundant router
>> >> > >>> configurations: Is using a setup_VPC_redundant_router
bash
>> function,
>> >> > >>> similar to the existing open network function mentioned
above, the
>> >> > >>> most appropriate way to setup the keepalived or conntrackd
>> >> > >>> configuration files for VPC redundant routers in the
>> >> > >>> cloud-early-script? It seems to me reading the parameters
from the
>> >> > >>> kernel will require a unwieldy set of kernels to match
the N
>> private
>> >> > >>> network redundant router pairs configured by the enduser.
>> >> > >>>
>> >> > >>>
>> >> > >>> Comments, questions, clarifications?
>> >> > >>
>> >> > >>>
>> >> > >>> Karl
>> >> > >>>
>> >> > >>>
>> >> > >>> In the bash script ea
>> >> > >>> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland <
>> >> > daan.hoogland@gmail.com>
>> >> > >> wrote:
>> >> > >>>> good reason to skip it for a next version, let's look
into it
>> >> anyway,
>> >> > >>>> as we don't want to burn any of our ships.
>> >> > >>>>
>> >> > >>>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris <
>> >> karl.harris@sungard.com
>> >> > >
>> >> > >> wrote:
>> >> > >>>>> All,
>> >> > >>>>>
>> >> > >>>>> At first redundant DHCP seemed like a good idea.
I did some
>> cursory
>> >> > >>>>> research and the more I read the more I'm convinced
it may be
>> >> > >>>>> more trouble than its worth for the first implementation.
I'll
>> talk
>> >> > >>>>> with some of our Systems Engineer's here and get
a broader
>> >> > >>>>> perspective.
>> >> > >>>>>
>> >> > >>>>> There seems to be only a single implementation
of an open source
>> >> DHCP
>> >> > >>>>> server that will handle the synchronization required
for
>> redundant
>> >> > >>>>> servers.
>> >> > >>>>>
>> >> > >>>>> Karl
>> >> > >>>>>
>> >> > >>>>>
>> >> > >>>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland
<
>> >> > >> daan.hoogland@gmail.com> wrote:
>> >> > >>>>>> Saurav,
>> >> > >>>>>>
>> >> > >>>>>> Not sure how this happens now, but it is definateluy
something
>> we
>> >> > >>>>>> want. For the static conf files it won't be
much of a problem.
>> The
>> >> > >>>>>> firewall/loadbalences won't be much of a problem,
they are fire
>> >> and
>> >> > >>>>>> forget configurations of the ms that can just
be sent to both.
>> The
>> >> > >>>>>> dhcp is a challange. I am not sure if it is
solved for the
>> plain
>> >> rvr
>> >> > >>>>>> now but it should be solved for that as well.
>> >> > >>>>>>
>> >> > >>>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri
>> >> > >>>>>> <saurav.lahiri@sungard.com> wrote:
>> >> > >>>>>>> Daan,
>> >> > >>>>>>> I was wondering if you had not shared
your thoughts, but looks
>> >> like
>> >> > >> I had
>> >> > >>>>>>> missed your mail.
>> >> > >>>>>>>
>> >> > >>>>>>> I agree that redundant dhcp or dns would
be good to have.
>> What I
>> >> > >> meant was,
>> >> > >>>>>>> it appears that by  just enabling RVR
  there is no way to
>> auto
>> >> > sync
>> >> > >>>>>>> configuration between the  master and
slave nodes with regard
>> to
>> >> > >> dhcp,
>> >> > >>>>>>> loadbalancer and firewall(specifically
the dhcp lease file,
>> >> > >> haproxy,cfg and
>> >> > >>>>>>> iptables configuration).  So just enabling
RVR does not ensure
>> >> high
>> >> > >>>>>>> availability for  these services. Is there
a way cloudstack
>> >> > autosyncs
>> >> > >>>>>>> configuration?
>> >> > >>>>>>>
>> >> > >>>>>>> For the routing portion this is not an
issue as the
>> participating
>> >> > >> routers
>> >> > >>>>>>> learn the route through known protocols.
>> >> > >>>>>>>
>> >> > >>>>>>> Thanks
>> >> > >>>>>>> Saurav
>> >> > >>>>>>>
>> >> > >>>>>>>
>> >> > >>>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan
Hoogland <
>> >> > >> daan.hoogland@gmail.com>wrote:
>> >> > >>>>>>>
>> >> > >>>>>>>> Saurav, I don't see why you can't
benefit from having other
>> >> > services
>> >> > >>>>>>>> redundant as well. Vpn might be a
problem as the source ip
>> of a
>> >> > >>>>>>>> redundant router pair on a vpn might
give a problem (maybe
>> there
>> >> > is
>> >> > >> an
>> >> > >>>>>>>> implementation) but why wouldn't you
want redundant dhcp or
>> dns
>> >> > >>>>>>>> services? As I understand it these
are used at Schuberg
>> Philis
>> >> at
>> >> > >> the
>> >> > >>>>>>>> moment. will double check when I get
a chance.
>> >> > >>>>>>>>
>> >> > >>>>>>>> regards,
>> >> > >>>>>>>> Daan
>> >> > >>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav
Lahiri
>> >> > >>>>>>>> <saurav.lahiri@sungard.com>
wrote:
>> >> > >>>>>>>>> Daan,
>> >> > >>>>>>>>> So looking at what is available
today for guest network, the
>> >> > >> Redundant VR
>> >> > >>>>>>>>> can be enabled only for the source
nat service. I guess the
>> >> > >> mention of
>> >> > >>>>>>>> the
>> >> > >>>>>>>>> source nat router is in relation
to that. I could be wrong
>> >> > though.
>> >> > >> It
>> >> > >>>>>>>>> appears  that the other services
like vpn, dhcp, dns do not
>> >> > >> benefit much
>> >> > >>>>>>>>> from the RVR capability. Can you
clarify if thats correct?
>> >> > >>>>>>>>>
>> >> > >>>>>>>>> Thanks
>> >> > >>>>>>>>> Saurav
>> >> > >>>>>>>>>
>> >> > >>>>>>>>>
>> >> > >>>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM,
Karl Harris <
>> >> > >> karl.harris@sungard.com
>> >> > >>>>>>>>> wrote:
>> >> > >>>>>>>>>
>> >> > >>>>>>>>>> ---------- Forwarded message
----------
>> >> > >>>>>>>>>> From: Daan Hoogland <daan.hoogland@gmail.com>
>> >> > >>>>>>>>>> Date: Wed, Jan 15, 2014 at
2:51 PM
>> >> > >>>>>>>>>> Subject: Re: rvr4vpc
>> >> > >>>>>>>>>> To: Karl Harris <karl.harris@sungard.com>
>> >> > >>>>>>>>>> Cc: Christopher Litsinger
<
>> christopher.litsinge@sungard.com>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> H Karl,
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> Thanks for sharing. I didn't
want to initiate this but I
>> >> > >> encourage you
>> >> > >>>>>>>>>> to share this on the dev list
(not in jira) as things are
>> only
>> >> > >>>>>>>>>> considered 'discussed' if
they passed by there.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> You speak of '1 Get configuration
data on Source Nat
>> Router',
>> >> I
>> >> > >> don't
>> >> > >>>>>>>>>> understand why you call the
router by this designation.
>> >> 'Source
>> >> > >> Nat'
>> >> > >>>>>>>>>> is only one of it's many possible
functions.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> Apart from the design principles
I shared with you I have
>> so
>> >> far
>> >> > >> only
>> >> > >>>>>>>>>> a technical implementation
detail so far. That is to
>> reserve
>> >> the
>> >> > >>>>>>>>>> (eth2) interface for the private
gateway on the vpc (r)vr.
>> >> This
>> >> > >> way
>> >> > >>>>>>>>>> the inteface to configure
are somewhat predictable.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> As for the design principle
to have a statefull router
>> (reboot
>> >> > >> proof)
>> >> > >>>>>>>>>> the idea is to implement a
configuration file that will be
>> >> > >> uploaded to
>> >> > >>>>>>>>>> the router after which a self-config
command is send that
>> will
>> >> > >>>>>>>>>> implement the details of configuring
the interfaces,
>> haproxy
>> >> and
>> >> > >>>>>>>>>> keepalived and maybe more.
I think your current assessment
>> of
>> >> > the
>> >> > >>>>>>>>>> working of the RVRs is accurate
but it will not be workable
>> >> for
>> >> > an
>> >> > >>>>>>>>>> implementation for vpc's as
they have an unpreditable
>> number
>> >> of
>> >> > >>>>>>>>>> interfaces.
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> to bad you can't make it next
thursday,
>> >> > >>>>>>>>>> Daan Hoogland
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> On Wed, Jan 15, 2014 at 3:25
PM, Karl Harris <
>> >> > >> karl.harris@sungard.com>
>> >> > >>>>>>>>>> wrote:
>> >> > >>>>>>>>>>> Daan,
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Sorry for the delayed
response but as Christopher
>> mentioned
>> >> to
>> >> > >> you in
>> >> > >>>>>>>> his
>> >> > >>>>>>>>>>> email I am getting my
head around the CloudStack software.
>> >> > >>>>>>>>>>> Since I am new to CloudStack
but "old" to enterprise level
>> >> JAVA
>> >> > >> the
>> >> > >>>>>>>> task
>> >> > >>>>>>>>>> is
>> >> > >>>>>>>>>>> large but not impossible.
I have no experience with
>> running
>> >> > >> CloudStack
>> >> > >>>>>>>>>> but
>> >> > >>>>>>>>>>> considerable experience
designing and maintaining large
>> JAVA
>> >> > >>>>>>>>>> applications.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> I've created what I believe
is a very high level abstract
>> of
>> >> > how
>> >> > >> the
>> >> > >>>>>>>>>> current
>> >> > >>>>>>>>>>> guest VRR's are created
for guest networks with the
>> intent of
>> >> > >> making
>> >> > >>>>>>>> this
>> >> > >>>>>>>>>>> abstract
>> >> > >>>>>>>>>>> more detailed.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> 1 Get configuration data
on Source Nat Router  selected
>> as a
>> >> > >> redundant
>> >> > >>>>>>>>>>> router
>> >> > >>>>>>>>>>>  1.1 Public and Guest
network identified.
>> >> > >>>>>>>>>>> 2 Both routers are provisioned
>> >> > >>>>>>>>>>>  2.1 Software  trys different,
>> >> > >> regions(?),zones,pods,clusters,hosts
>> >> > >>>>>>>> in
>> >> > >>>>>>>>>>> that order as the location
of the router. Log maximum
>> >> > "distance"
>> >> > >>>>>>>> apart.
>> >> > >>>>>>>>>>> 3 Keepalived is configured
>> >> > >>>>>>>>>>> 4 Both routers are started
>> >> > >>>>>>>>>>> 5 Keepalived is started
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Obviously there is much
more that is happening under each
>> of
>> >> > the
>> >> > >> steps
>> >> > >>>>>>>>>>> above. My intent is to
complete this detailed "as is"
>> listing
>> >> > as
>> >> > >> much
>> >> > >>>>>>>> as
>> >> > >>>>>>>>>> we
>> >> > >>>>>>>>>>> can. Then  using the "as
is" description/sequence
>> >> > >>>>>>>>>>> make a "to-be" addition
for VPC's. When I get a consensus
>> on
>> >> > WHAT
>> >> > >>>>>>>> needs
>> >> > >>>>>>>>>> to
>> >> > >>>>>>>>>>> be implemented for the
VRR in VPC  then develop HOW best
>> to
>> >> > >> implement
>> >> > >>>>>>>> the
>> >> > >>>>>>>>>>> "to-be" addition with
the
>> >> > >>>>>>>>>>> existing JAVA code as
well as what additional classes
>> need to
>> >> > be
>> >> > >>>>>>>>>>> extended/implemented/created.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Comments, critiques and
changes to the above sequence are
>> >> > >> encouraged.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> I plan on posting this
to the dev-list/Jira very soon.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> I have been using this
functional spec as a guide, after
>> >> > >> discussing
>> >> > >>>>>>>> this
>> >> > >>>>>>>>>>> with our Systems Engineering
people this spec meets our
>> >> > >> requirements.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Do you have an implementation
in mind?
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> We have an internal Cloud
Meeting with conflicts with the
>> >> > >> cloudstack
>> >> > >>>>>>>>>> "day"
>> >> > >>>>>>>>>>> next week so I will not
be in attendence.
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> Karl
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> On Tue, Jan 14, 2014 at
4:35 PM, Daan Hoogland <
>> >> > >>>>>>>> daan.hoogland@gmail.com>
>> >> > >>>>>>>>>>> wrote:
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> Hello overthere in
the states,
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> Tomorow I will start
some experimenting with redundant
>> vpc
>> >> > >> routers.
>> >> > >>>>>>>>>>>> This is to check up
on any findings and requirements that
>> >> you
>> >> > >> might
>> >> > >>>>>>>>>>>> have on this. Once
again I would not like to waste work
>> on
>> >> > this
>> >> > >> as it
>> >> > >>>>>>>>>>>> is really a globally
usable feature that is probably
>> >> > universal.
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> please let me know
your status on this.
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> If any of you are
coming to the cloudstack day in London
>> >> next
>> >> > >> week,
>> >> > >>>>>>>>>>>> let's meetup next
thursday.
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>> kind regards,
>> >> > >>>>>>>>>>>> Daan Hoogland
>> >> > >>>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>> --
>> >> > >>>>>>>>>>> Karl O. Harris
>> >> > >>>>>>>>>>> Cloud Software Engineer
>> >> > >>>>>>>>>>> Sungard Availability Services
>> >> > >>>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>>> --
>> >> > >>>>>>>>>> Karl O. Harris
>> >> > >>>>>>>>>> Cloud Software Engineer
>> >> > >>>>>>>>>> Sungard Availability Services
>> >> > >>>>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>>>
>> >> > >>>>>>
>> >> > >>>>>
>> >> > >>>>>
>> >> > >>>>>
>> >> > >>>>> --
>> >> > >>>>> Karl O. Harris
>> >> > >>>>> Cloud Software Engineer
>> >> > >>>>> Sungard Availability Services
>> >> > >>>>
>> >> > >>>
>> >> > >>>
>> >> > >>>
>> >> > >>> --
>> >> > >>> Karl O. Harris
>> >> > >>> Cloud Software Engineer
>> >> > >>> Sungard Availability Services
>> >> > >>
>> >> > >>
>> >> > >>
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Karl O. Harris
>> >> > > Cloud Software Engineer
>> >> > > Sungard Availability Services
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Karl O. Harris
>> >> Cloud Software Engineer
>> >> Sungard Availability Services
>> >>
>>
>>
>>
>> --
>> Daan
>>
>>
>
>
> --
> Karl O. Harris
> Cloud Software Engineer
> Sungard Availability Services



-- 
Daan

Mime
View raw message