cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daan Hoogland <>
Subject Re: Working on CloudStack Jira-764:nTier Apps 2.0 : Redundant Virtual Router for VPC email 2 of 2
Date Mon, 03 Feb 2014 17:54:37 GMT

On 03 Feb 2014, at 18:03, Karl Harris <> wrote:

> After discussion with my colleagues  questions about initial
> configuration of the open network redundant routers and the
> applicability of the existing bash scripts (cloud-early-config) to the
> setup of VPC redundant routers have generated.
> Some setup first: In the bash script cloud-early-config there is a
> function named setup_redundant_router which makes copies of several
> template files. The template files are used to configure keepalived
> and conntrackd. The template copies are edited, via sed editor, using
> environment variables ($ROUTER_PR, $ETH0_IP,$NAME, etc.)  which are
> obtained from the kernel of the current running linux image using the
> virtual file system /proc/cmdline.
> I'm sure keepalived and conntrackd  can be used for starting and
> control of VPC redundant routers. However the setup of keepalived and
> conntrackd for VPC needs setup parameters which are dynamic because a
> VPC can have N number of redundant router pairs, not just the fixed
> number parsed from proc/cmdline in the running kernel.
> Am I correct in this analysis?
Karl, I think not. There is only one router in a vac, it routes for all networks in the virtual
private cloud. Am I misreading your description?

> If so, given the dynamic nature of the VPC redundant router
> configurations: Is using a setup_VPC_redundant_router bash function,
> similar to the existing open network function mentioned above, the
> most appropriate way to setup the keepalived or conntrackd
> configuration files for VPC redundant routers in the
> cloud-early-script? It seems to me reading the parameters from the
> kernel will require a unwieldy set of kernels to match the N private
> network redundant router pairs configured by the enduser.
> Comments, questions, clarifications?

> Karl
> In the bash script ea
> On Mon, Jan 27, 2014 at 11:25 AM, Daan Hoogland <> wrote:
>> good reason to skip it for a next version, let's look into it anyway,
>> as we don't want to burn any of our ships.
>> On Mon, Jan 27, 2014 at 4:53 PM, Karl Harris <> wrote:
>>> All,
>>> At first redundant DHCP seemed like a good idea. I did some cursory
>>> research and the more I read the more I'm convinced it may be
>>> more trouble than its worth for the first implementation. I'll talk
>>> with some of our Systems Engineer's here and get a broader
>>> perspective.
>>> There seems to be only a single implementation of an open source DHCP
>>> server that will handle the synchronization required for redundant
>>> servers.
>>> Karl
>>> On Fri, Jan 24, 2014 at 5:47 PM, Daan Hoogland <>
>>>> Saurav,
>>>> Not sure how this happens now, but it is definateluy something we
>>>> want. For the static conf files it won't be much of a problem. The
>>>> firewall/loadbalences won't be much of a problem, they are fire and
>>>> forget configurations of the ms that can just be sent to both. The
>>>> dhcp is a challange. I am not sure if it is solved for the plain rvr
>>>> now but it should be solved for that as well.
>>>> On Fri, Jan 24, 2014 at 3:53 PM, Saurav Lahiri
>>>> <> wrote:
>>>>> Daan,
>>>>> I was wondering if you had not shared your thoughts, but looks like I
>>>>> missed your mail.
>>>>> I agree that redundant dhcp or dns would be good to have. What I meant
>>>>> it appears that by  just enabling RVR   there is no way to auto sync
>>>>> configuration between the  master and slave nodes with regard to dhcp,
>>>>> loadbalancer and firewall(specifically the dhcp lease file, haproxy,cfg
>>>>> iptables configuration).  So just enabling RVR does not ensure high
>>>>> availability for  these services. Is there a way cloudstack autosyncs
>>>>> configuration?
>>>>> For the routing portion this is not an issue as the participating routers
>>>>> learn the route through known protocols.
>>>>> Thanks
>>>>> Saurav
>>>>> On Fri, Jan 17, 2014 at 2:37 AM, Daan Hoogland <>wrote:
>>>>>> Saurav, I don't see why you can't benefit from having other services
>>>>>> redundant as well. Vpn might be a problem as the source ip of a
>>>>>> redundant router pair on a vpn might give a problem (maybe there
is an
>>>>>> implementation) but why wouldn't you want redundant dhcp or dns
>>>>>> services? As I understand it these are used at Schuberg Philis at
>>>>>> moment. will double check when I get a chance.
>>>>>> regards,
>>>>>> Daan
>>>>>> On Thu, Jan 16, 2014 at 1:33 PM, Saurav Lahiri
>>>>>> <> wrote:
>>>>>>> Daan,
>>>>>>> So looking at what is available today for guest network, the
Redundant VR
>>>>>>> can be enabled only for the source nat service. I guess the mention
>>>>>> the
>>>>>>> source nat router is in relation to that. I could be wrong though.
>>>>>>> appears  that the other services like vpn, dhcp, dns do not benefit
>>>>>>> from the RVR capability. Can you clarify if thats correct?
>>>>>>> Thanks
>>>>>>> Saurav
>>>>>>> On Thu, Jan 16, 2014 at 2:27 AM, Karl Harris <
>>>>>>> wrote:
>>>>>>>> ---------- Forwarded message ----------
>>>>>>>> From: Daan Hoogland <>
>>>>>>>> Date: Wed, Jan 15, 2014 at 2:51 PM
>>>>>>>> Subject: Re: rvr4vpc
>>>>>>>> To: Karl Harris <>
>>>>>>>> Cc: Christopher Litsinger <>
>>>>>>>> H Karl,
>>>>>>>> Thanks for sharing. I didn't want to initiate this but I
encourage you
>>>>>>>> to share this on the dev list (not in jira) as things are
>>>>>>>> considered 'discussed' if they passed by there.
>>>>>>>> You speak of '1 Get configuration data on Source Nat Router',
I don't
>>>>>>>> understand why you call the router by this designation. 'Source
>>>>>>>> is only one of it's many possible functions.
>>>>>>>> Apart from the design principles I shared with you I have
so far only
>>>>>>>> a technical implementation detail so far. That is to reserve
>>>>>>>> (eth2) interface for the private gateway on the vpc (r)vr.
This way
>>>>>>>> the inteface to configure are somewhat predictable.
>>>>>>>> As for the design principle to have a statefull router (reboot
>>>>>>>> the idea is to implement a configuration file that will be
uploaded to
>>>>>>>> the router after which a self-config command is send that
>>>>>>>> implement the details of configuring the interfaces, haproxy
>>>>>>>> keepalived and maybe more. I think your current assessment
of the
>>>>>>>> working of the RVRs is accurate but it will not be workable
for an
>>>>>>>> implementation for vpc's as they have an unpreditable number
>>>>>>>> interfaces.
>>>>>>>> to bad you can't make it next thursday,
>>>>>>>> Daan Hoogland
>>>>>>>> On Wed, Jan 15, 2014 at 3:25 PM, Karl Harris <>
>>>>>>>> wrote:
>>>>>>>>> Daan,
>>>>>>>>> Sorry for the delayed response but as Christopher mentioned
to you in
>>>>>> his
>>>>>>>>> email I am getting my head around the CloudStack software.
>>>>>>>>> Since I am new to CloudStack but "old" to enterprise
level JAVA the
>>>>>> task
>>>>>>>> is
>>>>>>>>> large but not impossible. I have no experience with running
>>>>>>>> but
>>>>>>>>> considerable experience designing and maintaining large
>>>>>>>> applications.
>>>>>>>>> I've created what I believe is a very high level abstract
of how the
>>>>>>>> current
>>>>>>>>> guest VRR's are created for guest networks with the intent
of making
>>>>>> this
>>>>>>>>> abstract
>>>>>>>>> more detailed.
>>>>>>>>> 1 Get configuration data on Source Nat Router  selected
as a redundant
>>>>>>>>> router
>>>>>>>>>   1.1 Public and Guest network identified.
>>>>>>>>> 2 Both routers are provisioned
>>>>>>>>>   2.1 Software  trys different, regions(?),zones,pods,clusters,hosts
>>>>>> in
>>>>>>>>> that order as the location of the router. Log maximum
>>>>>> apart.
>>>>>>>>> 3 Keepalived is configured
>>>>>>>>> 4 Both routers are started
>>>>>>>>> 5 Keepalived is started
>>>>>>>>> Obviously there is much more that is happening under
each of the steps
>>>>>>>>> above. My intent is to complete this detailed "as is"
listing as much
>>>>>> as
>>>>>>>> we
>>>>>>>>> can. Then  using the "as is" description/sequence
>>>>>>>>> make a "to-be" addition for VPC's. When I get a consensus
>>>>>> needs
>>>>>>>> to
>>>>>>>>> be implemented for the VRR in VPC  then develop HOW best
to implement
>>>>>> the
>>>>>>>>> "to-be" addition with the
>>>>>>>>> existing JAVA code as well as what additional classes
need to be
>>>>>>>>> extended/implemented/created.
>>>>>>>>> Comments, critiques and changes to the above sequence
are encouraged.
>>>>>>>>> I plan on posting this to the dev-list/Jira very soon.
>>>>>>>>> I have been using this functional spec as a guide, after
>>>>>> this
>>>>>>>>> with our Systems Engineering people this spec meets our
>>>>>>>>> Do you have an implementation in mind?
>>>>>>>>> We have an internal Cloud Meeting with conflicts with
the cloudstack
>>>>>>>> "day"
>>>>>>>>> next week so I will not be in attendence.
>>>>>>>>> Karl
>>>>>>>>> On Tue, Jan 14, 2014 at 4:35 PM, Daan Hoogland <
>>>>>>>>> wrote:
>>>>>>>>>> Hello overthere in the states,
>>>>>>>>>> Tomorow I will start some experimenting with redundant
vpc routers.
>>>>>>>>>> This is to check up on any findings and requirements
that you might
>>>>>>>>>> have on this. Once again I would not like to waste
work on this as it
>>>>>>>>>> is really a globally usable feature that is probably
>>>>>>>>>> please let me know your status on this.
>>>>>>>>>> If any of you are coming to the cloudstack day in
London next week,
>>>>>>>>>> let's meetup next thursday.
>>>>>>>>>> kind regards,
>>>>>>>>>> Daan Hoogland
>>>>>>>>> --
>>>>>>>>> Karl O. Harris
>>>>>>>>> Cloud Software Engineer
>>>>>>>>> Sungard Availability Services
>>>>>>>> --
>>>>>>>> Karl O. Harris
>>>>>>>> Cloud Software Engineer
>>>>>>>> Sungard Availability Services
>>> --
>>> Karl O. Harris
>>> Cloud Software Engineer
>>> Sungard Availability Services
> -- 
> Karl O. Harris
> Cloud Software Engineer
> Sungard Availability Services

View raw message