Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
To: dev@cloudstack.apache.org
References: <CAC2panpbVGqZNC_N97d0ojsF7mbU2HeNZ_3n5YpsB9AsZysP5w@mail.gmail.com>
 <CAC2panrCMQKH23gq+A16OVQ6KMYdpqmnnf2mt3dTF2BZfeyxsQ@mail.gmail.com>
 <BY2PR02MB20070849476D5C5A3E833021A9F00@BY2PR02MB2007.namprd02.prod.outlook.com>
 <CAC2panqRYP-icBrWojTWfCY+-XW251sMvW7ReU_mjP2yDTg-kQ@mail.gmail.com>
 <BY2PR02MB20074FF840D360F9F3BCF980A9F00@BY2PR02MB2007.namprd02.prod.outlook.com>
 <CAC2panrv3gnSAdRr5OtWiiG8ihaTtXJBTA5eFRmdAN6FH3OsUQ@mail.gmail.com>
 <987264713.37764.1474021206267.JavaMail.zimbra@li.nux.ro>
 <CAC2panpraVSZg=cMztBX1WgHfpKHnApTBi4ZGB=SzRLXu8=ZTw@mail.gmail.com>
 <CADMzvUq2qDsxi1rs0BPAegS6EFS0prkx=R89aU8pfStvYcmbqQ@mail.gmail.com>
 <CAC2panqRowyShQWJfVN=v1MqV=du7vk_2sZiwsVhK6_wzPbSzA@mail.gmail.com>
 <BN6PR14MB12661D7544CFB44340596EEEBDF30@BN6PR14MB1266.namprd14.prod.outlook.com>
 <BY2PR02MB2007986F337DA7AECFE5E73BA9F30@BY2PR02MB2007.namprd02.prod.outlook.com>
 <CAC2panpw2hDY6rcrVt-KjAThRA7-oj5kb4KXqV0ve-TFByW-0Q@mail.gmail.com>
From: ilya <ilya.mailing.lists@gmail.com>
Message-ID: <d6e958ff-6131-24bd-766d-f3850b8024fd@gmail.com>
Date: Sat, 17 Sep 2016 22:08:35 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0)
 Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <CAC2panpw2hDY6rcrVt-KjAThRA7-oj5kb4KXqV0ve-TFByW-0Q@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
archived-at: Sun, 18 Sep 2016 05:08:54 -0000

Our options become much better if we consider BSD based routers.

Would that be on the table?

https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions


On 9/16/16 12:04 PM, Will Stevens wrote:
> Ya, your points are all valid Simon.  The lack of standard libraries to
> handle a lot of the details is a problem.  I don't think it is an
> unsolvable problem, but if we spend the time to do that, will we have
> something that will work for us for the next 5 years?  This may be the
> shortest path to getting us where we need to be for the time being.
> 
> What is the best case scenario for the VR going forward which will last us
> the next 5 years?  Maybe we just clean up what we have to do a major
> restructuring of the pieces and how they are implemented.  We need to keep
> in mind how maintainable this implementation is because that is going to be
> key going forward IMO.
> 
> 
> 
> *Will STEVENS*
> Lead Developer
> 
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
> 
> On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sweller@ena.com> wrote:
> 
>> I think our other option is to take a real look at what it would take to
>> fix the VR. In my opinion, a lot of the problems are related to the
>> monolithic python code base and the fact nothing is actually separated.
>>
>> Secondly, the python scripts (and bash scripts) don't use any established
>> libraries to complete tasks and instead shell out and run commands that are
>> both hard to track and hard to parse on return.
>>
>>
>> If we daemonized this, used a real api for Agent to VR communication, used
>> common already existing libraries for the system service and network
>> interactions and spent a bit of time separating out code into distinct
>> modules, everything would behave a lot better.
>>
>>
>> The pain and suffering is due to years and years of patches and constant
>> shelling out to complete tasks in my opinion. If we spend time to rethink
>> how we interact with the VR in general and we abstract the systems and
>> networking stuff and use well known and stable libraries to do the work,
>> the VR would be much easier to maintain.
>>
>>
>> - Si
>>
>>
>>
>>
>> ________________________________
>> From: Marty Godsey <marty@gonsource.com>
>> Sent: Friday, September 16, 2016 12:24 PM
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] Replacing the VR
>>
>> So based upon this discussion would it be prudent to wait on VyOS 2.0? The
>> current VR is giving us issues but would the time invested in another
>> "solution" be wasted especially if by the time another option is chose,
>> then coded, then tested, then implemented and right as that time happened
>> to be when VyOS 2.0 is released.  Of course you said they are just in the
>> scoping range so this could still be a year or more out.
>>
>> Thoughts?
>>
>> Regards,
>> Marty Godsey
>> nSource Solutions
>>
>> -----Original Message-----
>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>> Sent: Friday, September 16, 2016 10:31 AM
>> To: dev@cloudstack.apache.org
>> Cc: daniil@baturin.org
>> Subject: Re: [DISCUSS] Replacing the VR
>>
>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>> I have CC'ed one of them in case we have more licensing questions.
>>
>> So here is the status with the license "the code inherited from Vyatta and
>> our modifications from it is GPLv2 (strict, not v2+). The config reading
>> library is GPLv2 too, so anything that links to is is GPLv2.
>> Some auxiliary components we made after the fork are more permissive,
>> LGPLv2+ or MIT."
>>
>> They are currently in the process of scoping a redesign (VyOS 2.0), "we
>> are planning a clean rewrite that will solve issues of the current config
>> system".
>> This will include the ability to configure via the API.
>>
>> If we have more questions for VyOS, they are very friendly and responsive,
>> so we should be able to get answers.
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>>
>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sahmed@cloudops.com> wrote:
>>
>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>> Most of the outages we've had recently have somehow involved the VR.
>>> We set custom iptables rules on the VR which can and have easily gone
>> wrong.
>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>> VVRP with redundant router still needs work, and not to mention the
>>> problems we will have when we introduce IPv6 into the whole picture.
>>>
>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>> the networking for us (eg VyOS) and have us handle just the
>>> orchestration. All the problems that I've described have already been
>>> solved in VyOS. We also get the advantage of a potential wider
>>> community to fix and maintain the VR and given our current development
>>> velocity, it think it totally makes sense to look for a 3rd party option.
>>>
>>> -Syed
>>>
>>>
>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <wstevens@cloudops.com>
>>> wrote:
>>>
>>>> The VR has been biting us far too often recently, which is why we
>>>> have started looking into alternative implementations.
>>>>
>>>> One of the things that is nice about potentially using the VyOS is
>>>> that
>>> it
>>>> is based on Debian, so we should be able to run the other services
>>>> that
>>> we
>>>> currently have like the password server and userdata on the VyOS.
>>>> This means we would not have to change our architecture initially
>>>> and could focus on only replacing the networking paths.
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>> tw @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nux@li.nux.ro> wrote:
>>>>
>>>>> The more this is discussed the more I think we should stick with
>>>>> our
>>> VR.
>>>>>
>>>>> All these other options either seem unfinished or with
>>>>> incompatible license.
>>>>>
>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>> Adopting it though means we'll have to microservice our way out of
>>>>> it
>>>> with
>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>> those
>>>>> too. Imho this adds complexity we should void.
>>>>>
>>>>> --
>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>
>>>>> Nux!
>>>>> www.nux.ro
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Will Stevens" <wstevens@cloudops.com>
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>>> Ya, we would need to add a daemon for VPN as well.  Load
>>>>>> balancing is another aspect which we will need to consider if we
>> went this route.
>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>> fit
>>> due
>>>>> to
>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>
>>>>>> We should probably try define which pieces make sense to be
>>>>>> solved
>>>>> together
>>>>>> and which pieces would be best suited to be broken out.
>>>>>>
>>>>>> I think the network connectivity, routing and firewalling should
>>>> probably
>>>>>> all stay together since the majority of the tools we would
>>> potentially
>>>>> use
>>>>>> would handle all of that together in a single implementation.
>>>>>>
>>>>>> The password server and userdata seems like a good option for
>>>>>> being
>>>>> broken
>>>>>> out and handled independently (and probably rewritten completely
>>> since
>>>>> they
>>>>>> currently have some issues).
>>>>>>
>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>> that depends on what direction we go and how we would be managing
>> it.
>>> DHCP
>>>>> and
>>>>>> DNS are others which could go either way.
>>>>>>
>>>>>> If we do split out services, I think we should consolidate as
>>>>>> much as
>>>> we
>>>>>> can into each service we break out.  Ideally a network packet
>>>>>> would
>>>> never
>>>>>> hit more than one, maybe two, services.  I don't think we should
>>>>>> be splitting services 'just because', I think we need a valid
>>>>>> case for splitting any service out because it adds complexity.
>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>> complexity unless it
>>> is
>>>>>> really needed.
>>>>>>
>>>>>> Some more of my thoughts on this anyway...
>>>>>>
>>>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>> *|* tw @CloudOps_
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sweller@ena.com>
>>>> wrote:
>>>>>>
>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>> the
>>>>> password
>>>>>>> service and user data.
>>>>>>>
>>>>>>>
>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>> it
>>>>> doesn't
>>>>>>> seem that difficult to add new daemons:
>>> https://opensnaproute.github.
>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>
>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>> creating-new-component>
>>>>>>>
>>>>>>>
>>>>>>> They've definitely build it with a microservices architecture
>>>>>>> in
>>> mind,
>>>>> so
>>>>>>> each individual feature is abstracted into it's own small
>>>>>>> daemon
>>>>> process.
>>>>>>> We could just create a daemon for the password server and the
>>> userdata
>>>>>>> components if we really had to.
>>>>>>>
>>>>>>>
>>>>>>> - Si
>>>>>>>
>>>>>>>
>>>>>>> ________________________________
>>>>>>> From: williamstevens@gmail.com <williamstevens@gmail.com> on
>>>>>>> behalf
>>>> of
>>>>>>> Will Stevens <wstevens@cloudops.com>
>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>
>>>>>>> A big part of why I know about it is because it is written in Go.
>>> :P
>>>>>>>
>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>> handling
>>>>>>> aspects of the VR.  We will likely have to rethink some of the
>>> pieces
>>>> a
>>>>>>> little bit like the password server and userdata if we are to
>>>>>>> adopt
>>> a
>>>>>>> different VR approach.  This is where I think some of JohnB and
>>>>> Chiradeep's
>>>>>>> ideas make sense.  In many ways, it does not make sense for the
>>> device
>>>>>>> handling routing and network traffic to also be responsible for
>>>>> passwords
>>>>>>> and userdata.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>> *|* tw @CloudOps_
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sweller@ena.com>
>>>> wrote:
>>>>>>>
>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>> pretty
>>>>>>> cool!
>>>>>>>> It even supports ONIE install.
>>>>>>>>
>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>> offload
>>>>> it to
>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>> from a
>>>>>>> routing
>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>
>>>>>>>>
>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>> capabilities
>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>> will
>>> be
>>>>> huge
>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>> do
>>>>> private
>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>> stuff
>>>> on
>>>>> our
>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>> able to
>>>> do
>>>>>>> MPLS
>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>
>>>>>>>>
>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>> know
>>>> very
>>>>>>>> well).
>>>>>>>>
>>>>>>>>
>>>>>>>> - Si
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: Will Stevens <williamstevens@gmail.com>
>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>
>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>> does
>>>>> cover is
>>>>>>>> all api driven...
>>>>>>>>
>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <marty@gonsource.com>
>>>> wrote:
>>>>>>>>
>>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
>>>>>>>>> was
>>> not
>>>>>>>>> intended to do IPSec.
>>>>>>>>>
>>>>>>>>> It seems as though VyOS is starting to look like the best
>>> option.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Marty Godsey
>>>>>>>>> nSource Solutions
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>> [mailto:williamstevens@gmail.com
>>> ]
>>>> On
>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Or we could go completely crazy and go with something like
>>>>> FlexSwitch
>>>>>>>> from
>>>>>>>>> SnapRoute
>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>> cloudops.com
>>>> *|*
>>>>> tw
>>>>>>>>> @CloudOps_
>>>>>>>>>
>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>> wstevens@cloudops.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I tend to agree with Syed and Marty.  I am not sure what
>>>> problems
>>>>> are
>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>> bunch of
>>>>>>> separate
>>>>>>>>>> services.  As Syed points out, the complexity added is
>>>>> non-trivial.
>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>> as
>>> well
>>>> as
>>>>>>> the
>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>
>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>> of
>>> our
>>>>> use
>>>>>>>>>> case with a single unified control plane.  It also has
>>>>>>>>>> good
>>>>> support
>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>> cloudops.com
>>>>> *|*
>>>>>>> tw
>>>>>>>>>> @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>> sahmed@cloudops.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>> picture
>>>>> although
>>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>> is
>>> just
>>>>> not
>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>> networking
>>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>> are
>>>> very
>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>
>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>> does
>>>> not
>>>>>>>>>>> provide an api but it does fit the bill quite well
>>> otherwise. I
>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>> model
>>>> and
>>>>>>> you
>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>> marty@gonsource.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>> functions
>>> into
>>>>>>>>>>>> containers is not a good route either. This will force
>>> users
>>>> to
>>>>>>>>>>>> have to maintain
>>>>>>>>>>> and
>>>>>>>>>>>> use containers and adds complexity to the networking
>>> aspects
>>>> of
>>>>>>> ACS.
>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>> argument
>>>>> that
>>>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>> issues but
>>>> it
>>>>>>> also
>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>> of
>>> the
>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>>  - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>> touch
>>>>> anything
>>>>>>>>>>>> that
>>>>>>>>>>> has
>>>>>>>>>>>> AGPL
>>>>>>>>>>>>  - the github repo shows rather infrequent updates.
>>>>>>>>>>>> Quite
>>>>> likely
>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>> CloudStack
>>>>> community
>>>>>>>>>>>>
>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>> Split
>>> it
>>>>> into
>>>>>>>>>>>> many docker containers
>>>>>>>>>>>>  - password server
>>>>>>>>>>>>  - userdata server
>>>>>>>>>>>>  - DHCP / DNS
>>>>>>>>>>>>  - s2s VPN
>>>>>>>>>>>>  - RA VPN
>>>>>>>>>>>>  - intra-VPC routing and ACL
>>>>>>>>>>>>  - Port forwarding + NAT
>>>>>>>>>>>>  - FW
>>>>>>>>>>>>  - LB (public)
>>>>>>>>>>>>  - LB (internal),
>>>>>>>>>>>>  - secondary storage
>>>>>>>>>>>>  - agent
>>>>>>>>>>>> Glue them together with  docker compose files (one per
>>>>>>>>>>>> use
>>>>> case -
>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>
>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>> test
>>> each
>>>> of
>>>>>>> the
>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>> field
>>> (say
>>>>>>> DHCP)
>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>> build per-hypervisor VRs. You could even run on
>> baremetal.
>>>>>>>>>>>>
>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>  - make the traffic traverse the containers that are
>>>>>>>>>>>> needed
>>>> to
>>>>> be
>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>  - bootstrap the router (how does it find its compose
>> file?
>>>>> where
>>>>>>>>>>>> is the
>>>>>>>>>>>> registry?)
>>>>>>>>>>>>  - rethink the command and control of the VR
>>>>>>>>>>>> functions. SSH
>>>>> works,
>>>>>>>>>>>> but something more declarative, idempotent should be
>>>> explored.
>>>>>>>>>>>>
>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>> functions
>>> can
>>>>> be
>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>> Control
>>>> of
>>>>> the
>>>>>>>>>>> docker
>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>> <marty@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>> lack
>>> of
>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>> do not
>>>>> have
>>>>>>> an
>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>> save
>>>>> future
>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF
>> etc.
>>>>>>>>>>>>> And the best part is they come with test automation
>>>>> framework.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>> <jayapal.uradi@accelerite.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>> (on
>>>>> upgrade)
>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>> we
>>> want
>>>> at
>>>>>>>>>>>>> that release
>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>> them
>>>> are
>>>>>>>>>>>>>> running
>>>>>>>>>>>>> without issues.  When we replicate the ACS VR
>>>>>>>>>>>>> features in
>>>> new
>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>> (hidden
>>>>> bugs).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>> keen
>>> on
>>>>> VyOS
>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>>>> @shapeblue
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>> From: "Will Stevens" <williamstevens@gmail.com>
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>> should
>>>> be
>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this
>> idea...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>> potentially
>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>> Source
>>>>> Vyatta
>>>>>>>> VM).
>>>>>>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>> is
>>>>> licensed
>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>> assume
>>>>> we
>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>> and I
>>>> have
>>>>> to
>>>>>>>>>>>>>>>> admit, I was pretty impressed.  It is simple and
>>>>> intuitive
>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>> the
>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>> simpler
>>>>> more
>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>> configuration
>>>>>>>>> interface.
>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>> VRRP.
>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
>>>>>>>>>>>>>>>> rollback
>>> on
>>>>>>> error.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>> implementation.
>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>> consider
>>>>> this
>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>> Another option, which I don't know as well,
>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>> information
>>>>>>>>>>>>>> which is
>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>> business.
>>>>> It is
>>>>>>>>>>>>> intended only for the use of the individual or
>>>>>>>>>>>>> entity to
>>>>> which
>>>>>>> it
>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>> you
>>>> are
>>>>> not
>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
>>>>>>>>>>>>> or
>>> use
>>>>> this
>>>>>>>>>>>>> message. If you have received this communication in
>>> error,
>>>>>>> please
>>>>>>>>>>>>> notify the sender and delete all copies of this
>> message.
>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>> accept
>>>> any
>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>> infected mails.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>
>>>>
>>>
>>
>