cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daan Hoogland <daan.hoogl...@gmail.com>
Subject Re: Working on CloudStack Jira-764:nTier Apps 2.0 : Redundant Virtual Router for VPC email 2 of 2
Date Mon, 03 Feb 2014 19:58:47 GMT
On Mon, Feb 3, 2014 at 8:50 PM, Sheng Yang <sheng@yasker.org> wrote:
> On Mon, Feb 3, 2014 at 11:27 AM, Karl Harris <karl.harris@sungard.com>wrote:
>
>> On Mon, Feb 3, 2014 at 2:03 PM, Daan Hoogland <daan.hoogland@gmail.com
>> >wrote:
>>
>> >
>> > On 03 Feb 2014, at 19:45, Karl Harris <karl.harris@sungard.com> wrote:
>> >
>> > > On Mon, Feb 3, 2014 at 12:54 PM, Daan Hoogland <
>> daan.hoogland@gmail.com
>> > >wrote:
>> > >
>> > >>
>> > >> On 03 Feb 2014, at 18:03, Karl Harris <karl.harris@sungard.com>
>> wrote:
>> > >>
>> > >>> After discussion with my colleagues  questions about initial
>> > >>> configuration of the open network redundant routers and the
>> > >>> applicability of the existing bash scripts (cloud-early-config)
to
>> the
>> > >>> setup of VPC redundant routers have generated.
>> > >>>
>> > >>>
>> > >>> Some setup first: In the bash script cloud-early-config there is
a
>> > >>> function named setup_redundant_router which makes copies of several
>> > >>> template files. The template files are used to configure keepalived
>> > >>> and conntrackd. The template copies are edited, via sed editor,
using
>> > >>> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.)  which
are
>> > >>> obtained from the kernel of the current running linux image using
the
>> > >>> virtual file system /proc/cmdline.
>> > >>>
>> > >>> I'm sure keepalived and conntrackd  can be used for starting and
>> > >>> control of VPC redundant routers. However the setup of keepalived
and
>> > >>> conntrackd for VPC needs setup parameters which are dynamic because
a
>> > >>> VPC can have N number of redundant router pairs, not just the fixed
>> > >>> number parsed from proc/cmdline in the running kernel.
>> > >>>
>> > >>> Am I correct in this analysis?
>> > >> Karl, I think not. There is only one router in a vac, it routes for
>> all
>> > >> networks in the virtual private cloud. Am I misreading your
>> description.
>> > >
>> > > Ok then a VPC should contain a single router containing N+1 (N+2 if
>> > using a
>> > > VPN connection) nic's? Where N is the number of private networks.
>> > And I would like to see N+3 as it will allocate another when a private
>> > gateway is defined. That is why I proposed to pre-allocate this nic to be
>> > able to predict mic-ids, i.e. eth<#>.
>> >
>> > >
>> > > If so then my original question still holds in the sense that the nic's
>> > > need to be created in the VM kernel based on the number of private
>> > networks
>> > > desired?
>> >
>> > Yes
>> >
>> > > I found the method update_System_Vm_Templates in the jar file
>> > > Upgrade410to420 in cloud-engine-schema/src/com/cloud/upgrade/dao which
>> > > seems to point to a set of pre-configured VM images.
>> > I can not find the method you describe but these preconfigured images are
>> > the ones used for all the system vms. That would be secondary storage
>> vms,
>> > console proxy vms and indeed the router vms.
>> >
>> > Based on the discussion above the the configuration part of the VPC
>> redundant router is:
>>
>>  1. Collecting the correct data required to configure the nics. (Is this
>> part already in the VPC setup code? I will look for it, pointers will be
>> helpful.)
>>
>
> vpc_guestgw.sh
and VpcVirtualNetworkApplianceManager and -Impl and their ancestors


>
>
>>  2. Based on the hypervisor/vm types configure the nic(s) on the router VM
>> using the data collected above (Same question as in 1 above is this code
>> already in place?). My guess is to start with a preconfigured template and
>
> configure the vm based on the number of nics and any other appropriate data.
>>
>
> Yes.
>
>
>>  3. Configuring conntrackd and keepalived after the router vm (nics) have
>> been configured and the router(s) started or restarted? I still need to
>> work through this.
>>
>
> Likely you would need this.
>
> Don't know if keepalived can be start if there is no virtual IP definition.
>
> You also need to think about what to do second router(or the router pair)
> when the last tier of VPC has been destroyed.
>
> --Sheng
>
>
>>
>>
>>
>>
>> >
>> > >>
>> > >>> If so, given the dynamic nature of the VPC redundant router
>> > >>> configurations: Is using a setup_VPC_redundant_router bash function,
>> > >>> similar to the existing open network function mentioned above,
the
>> > >>> most appropriate way to setup the keepalived or conntrackd
>> > >>> configuration files for VPC redundant routers in the
>> > >>> cloud-early-script? It seems to me reading the parameters from
the
>> > >>> kernel will require a unwieldy set of kernels to match the N private
>> > >>> network redundant router pairs configured by the enduser.
>> > >>>
>> > >>>
>> > >>> Comments, questions, clarifications?
>> > >>
>> > >>>
>> > >>> Karl
>> > >>>
>> > >>>
>> > >>> In the bash script ea
>> > >>> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland <
>> > daan.hoogland@gmail.com>
>> > >> wrote:
>> > >>>> good reason to skip it for a next version, let's look into
it
>> anyway,
>> > >>>> as we don't want to burn any of our ships.
>> > >>>>
>> > >>>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris <
>> karl.harris@sungard.com
>> > >
>> > >> wrote:
>> > >>>>> All,
>> > >>>>>
>> > >>>>> At first redundant DHCP seemed like a good idea. I did
some cursory
>> > >>>>> research and the more I read the more I'm convinced it
may be
>> > >>>>> more trouble than its worth for the first implementation.
I'll talk
>> > >>>>> with some of our Systems Engineer's here and get a broader
>> > >>>>> perspective.
>> > >>>>>
>> > >>>>> There seems to be only a single implementation of an open
source
>> DHCP
>> > >>>>> server that will handle the synchronization required for
redundant
>> > >>>>> servers.
>> > >>>>>
>> > >>>>> Karl
>> > >>>>>
>> > >>>>>
>> > >>>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland <
>> > >> daan.hoogland@gmail.com> wrote:
>> > >>>>>> Saurav,
>> > >>>>>>
>> > >>>>>> Not sure how this happens now, but it is definateluy
something we
>> > >>>>>> want. For the static conf files it won't be much of
a problem. The
>> > >>>>>> firewall/loadbalences won't be much of a problem, they
are fire
>> and
>> > >>>>>> forget configurations of the ms that can just be sent
to both. The
>> > >>>>>> dhcp is a challange. I am not sure if it is solved
for the plain
>> rvr
>> > >>>>>> now but it should be solved for that as well.
>> > >>>>>>
>> > >>>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri
>> > >>>>>> <saurav.lahiri@sungard.com> wrote:
>> > >>>>>>> Daan,
>> > >>>>>>> I was wondering if you had not shared your thoughts,
but looks
>> like
>> > >> I had
>> > >>>>>>> missed your mail.
>> > >>>>>>>
>> > >>>>>>> I agree that redundant dhcp or dns would be good
to have. What I
>> > >> meant was,
>> > >>>>>>> it appears that by  just enabling RVR   there is
no way to auto
>> > sync
>> > >>>>>>> configuration between the  master and slave nodes
with regard to
>> > >> dhcp,
>> > >>>>>>> loadbalancer and firewall(specifically the dhcp
lease file,
>> > >> haproxy,cfg and
>> > >>>>>>> iptables configuration).  So just enabling RVR
does not ensure
>> high
>> > >>>>>>> availability for  these services. Is there a way
cloudstack
>> > autosyncs
>> > >>>>>>> configuration?
>> > >>>>>>>
>> > >>>>>>> For the routing portion this is not an issue as
the participating
>> > >> routers
>> > >>>>>>> learn the route through known protocols.
>> > >>>>>>>
>> > >>>>>>> Thanks
>> > >>>>>>> Saurav
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan Hoogland
<
>> > >> daan.hoogland@gmail.com>wrote:
>> > >>>>>>>
>> > >>>>>>>> Saurav, I don't see why you can't benefit from
having other
>> > services
>> > >>>>>>>> redundant as well. Vpn might be a problem as
the source ip of a
>> > >>>>>>>> redundant router pair on a vpn might give a
problem (maybe there
>> > is
>> > >> an
>> > >>>>>>>> implementation) but why wouldn't you want redundant
dhcp or dns
>> > >>>>>>>> services? As I understand it these are used
at Schuberg Philis
>> at
>> > >> the
>> > >>>>>>>> moment. will double check when I get a chance.
>> > >>>>>>>>
>> > >>>>>>>> regards,
>> > >>>>>>>> Daan
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav Lahiri
>> > >>>>>>>> <saurav.lahiri@sungard.com> wrote:
>> > >>>>>>>>> Daan,
>> > >>>>>>>>> So looking at what is available today for
guest network, the
>> > >> Redundant VR
>> > >>>>>>>>> can be enabled only for the source nat
service. I guess the
>> > >> mention of
>> > >>>>>>>> the
>> > >>>>>>>>> source nat router is in relation to that.
I could be wrong
>> > though.
>> > >> It
>> > >>>>>>>>> appears  that the other services like vpn,
dhcp, dns do not
>> > >> benefit much
>> > >>>>>>>>> from the RVR capability. Can you clarify
if thats correct?
>> > >>>>>>>>>
>> > >>>>>>>>> Thanks
>> > >>>>>>>>> Saurav
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM, Karl Harris
<
>> > >> karl.harris@sungard.com
>> > >>>>>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> ---------- Forwarded message ----------
>> > >>>>>>>>>> From: Daan Hoogland <daan.hoogland@gmail.com>
>> > >>>>>>>>>> Date: Wed, Jan 15, 2014 at 2:51 PM
>> > >>>>>>>>>> Subject: Re: rvr4vpc
>> > >>>>>>>>>> To: Karl Harris <karl.harris@sungard.com>
>> > >>>>>>>>>> Cc: Christopher Litsinger <christopher.litsinge@sungard.com>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> H Karl,
>> > >>>>>>>>>>
>> > >>>>>>>>>> Thanks for sharing. I didn't want to
initiate this but I
>> > >> encourage you
>> > >>>>>>>>>> to share this on the dev list (not
in jira) as things are only
>> > >>>>>>>>>> considered 'discussed' if they passed
by there.
>> > >>>>>>>>>>
>> > >>>>>>>>>> You speak of '1 Get configuration data
on Source Nat Router',
>> I
>> > >> don't
>> > >>>>>>>>>> understand why you call the router
by this designation.
>> 'Source
>> > >> Nat'
>> > >>>>>>>>>> is only one of it's many possible functions.
>> > >>>>>>>>>>
>> > >>>>>>>>>> Apart from the design principles I
shared with you I have so
>> far
>> > >> only
>> > >>>>>>>>>> a technical implementation detail so
far. That is to reserve
>> the
>> > >>>>>>>>>> (eth2) interface for the private gateway
on the vpc (r)vr.
>> This
>> > >> way
>> > >>>>>>>>>> the inteface to configure are somewhat
predictable.
>> > >>>>>>>>>>
>> > >>>>>>>>>> As for the design principle to have
a statefull router (reboot
>> > >> proof)
>> > >>>>>>>>>> the idea is to implement a configuration
file that will be
>> > >> uploaded to
>> > >>>>>>>>>> the router after which a self-config
command is send that will
>> > >>>>>>>>>> implement the details of configuring
the interfaces, haproxy
>> and
>> > >>>>>>>>>> keepalived and maybe more. I think
your current assessment of
>> > the
>> > >>>>>>>>>> working of the RVRs is accurate but
it will not be workable
>> for
>> > an
>> > >>>>>>>>>> implementation for vpc's as they have
an unpreditable number
>> of
>> > >>>>>>>>>> interfaces.
>> > >>>>>>>>>>
>> > >>>>>>>>>> to bad you can't make it next thursday,
>> > >>>>>>>>>> Daan Hoogland
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Wed, Jan 15, 2014 at 3:25 PM, Karl
Harris <
>> > >> karl.harris@sungard.com>
>> > >>>>>>>>>> wrote:
>> > >>>>>>>>>>> Daan,
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Sorry for the delayed response
but as Christopher mentioned
>> to
>> > >> you in
>> > >>>>>>>> his
>> > >>>>>>>>>>> email I am getting my head around
the CloudStack software.
>> > >>>>>>>>>>> Since I am new to CloudStack but
"old" to enterprise level
>> JAVA
>> > >> the
>> > >>>>>>>> task
>> > >>>>>>>>>> is
>> > >>>>>>>>>>> large but not impossible. I have
no experience with running
>> > >> CloudStack
>> > >>>>>>>>>> but
>> > >>>>>>>>>>> considerable experience designing
and maintaining large JAVA
>> > >>>>>>>>>> applications.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> I've created what I believe is
a very high level abstract of
>> > how
>> > >> the
>> > >>>>>>>>>> current
>> > >>>>>>>>>>> guest VRR's are created for guest
networks with the intent of
>> > >> making
>> > >>>>>>>> this
>> > >>>>>>>>>>> abstract
>> > >>>>>>>>>>> more detailed.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> 1 Get configuration data on Source
Nat Router  selected as a
>> > >> redundant
>> > >>>>>>>>>>> router
>> > >>>>>>>>>>>  1.1 Public and Guest network identified.
>> > >>>>>>>>>>> 2 Both routers are provisioned
>> > >>>>>>>>>>>  2.1 Software  trys different,
>> > >> regions(?),zones,pods,clusters,hosts
>> > >>>>>>>> in
>> > >>>>>>>>>>> that order as the location of the
router. Log maximum
>> > "distance"
>> > >>>>>>>> apart.
>> > >>>>>>>>>>> 3 Keepalived is configured
>> > >>>>>>>>>>> 4 Both routers are started
>> > >>>>>>>>>>> 5 Keepalived is started
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Obviously there is much more that
is happening under each of
>> > the
>> > >> steps
>> > >>>>>>>>>>> above. My intent is to complete
this detailed "as is" listing
>> > as
>> > >> much
>> > >>>>>>>> as
>> > >>>>>>>>>> we
>> > >>>>>>>>>>> can. Then  using the "as is" description/sequence
>> > >>>>>>>>>>> make a "to-be" addition for VPC's.
When I get a consensus on
>> > WHAT
>> > >>>>>>>> needs
>> > >>>>>>>>>> to
>> > >>>>>>>>>>> be implemented for the VRR in VPC
 then develop HOW best to
>> > >> implement
>> > >>>>>>>> the
>> > >>>>>>>>>>> "to-be" addition with the
>> > >>>>>>>>>>> existing JAVA code as well as what
additional classes need to
>> > be
>> > >>>>>>>>>>> extended/implemented/created.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Comments, critiques and changes
to the above sequence are
>> > >> encouraged.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> I plan on posting this to the dev-list/Jira
very soon.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> I have been using this functional
spec as a guide, after
>> > >> discussing
>> > >>>>>>>> this
>> > >>>>>>>>>>> with our Systems Engineering people
this spec meets our
>> > >> requirements.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Do you have an implementation in
mind?
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> We have an internal Cloud Meeting
with conflicts with the
>> > >> cloudstack
>> > >>>>>>>>>> "day"
>> > >>>>>>>>>>> next week so I will not be in attendence.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Karl
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Tue, Jan 14, 2014 at 4:35 PM,
Daan Hoogland <
>> > >>>>>>>> daan.hoogland@gmail.com>
>> > >>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Hello overthere in the states,
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Tomorow I will start some experimenting
with redundant vpc
>> > >> routers.
>> > >>>>>>>>>>>> This is to check up on any
findings and requirements that
>> you
>> > >> might
>> > >>>>>>>>>>>> have on this. Once again I
would not like to waste work on
>> > this
>> > >> as it
>> > >>>>>>>>>>>> is really a globally usable
feature that is probably
>> > universal.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> please let me know your status
on this.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> If any of you are coming to
the cloudstack day in London
>> next
>> > >> week,
>> > >>>>>>>>>>>> let's meetup next thursday.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> kind regards,
>> > >>>>>>>>>>>> Daan Hoogland
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> --
>> > >>>>>>>>>>> Karl O. Harris
>> > >>>>>>>>>>> Cloud Software Engineer
>> > >>>>>>>>>>> Sungard Availability Services
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> --
>> > >>>>>>>>>> Karl O. Harris
>> > >>>>>>>>>> Cloud Software Engineer
>> > >>>>>>>>>> Sungard Availability Services
>> > >>>>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> --
>> > >>>>> Karl O. Harris
>> > >>>>> Cloud Software Engineer
>> > >>>>> Sungard Availability Services
>> > >>>>
>> > >>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> Karl O. Harris
>> > >>> Cloud Software Engineer
>> > >>> Sungard Availability Services
>> > >>
>> > >>
>> > >>
>> > >
>> > >
>> > > --
>> > > Karl O. Harris
>> > > Cloud Software Engineer
>> > > Sungard Availability Services
>> >
>> >
>>
>>
>> --
>> Karl O. Harris
>> Cloud Software Engineer
>> Sungard Availability Services
>>



-- 
Daan

Mime
View raw message