cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Smart <msm...@smartsoftwareinc.com>
Subject Re: [DISCUSS] Replacing the VR
Date Mon, 19 Sep 2016 17:59:49 GMT
Abstracting that interaction would also make future implementations MUCH 
less painful and allows third parties who want their networking products 
to be CS compatible to write their own implementation of a published API 
that CS defines. Let them do the legwork for their products, expand CS 
compatibility, and ease future refactors of VR all in one package. I 
like it.

Caveat: I am clueless about how VR configuration happens under the hood 
currently or if abstracting it is a monumental task.

Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501

Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com

On 09/19/2016 11:50 AM, Paul Angus wrote:
> Hi All,
>
>  From the looks of this thread and the different preferences that parties have, the way forward looks to be a model that allows multiple different VRs to be used is the answer.
>
> This is something that I have had in mind for some time now.
>
> By abstracting the roles and configuration of the VR, drivers can be written which transform requests such as 'add these firewall rules' into something that a VyOs, Fortigate, pfSense, Cisco ASAv, a Juniper vSRX or a homebrew VR can understand and implement.
>
> Ideally I'd like to see the features separated and service chaining introduced such that someone might use a stock VR with a VPN appliance or a Cisco ASAv in front of a VR doing load balancing.
>
> This moves us forward into a much more NFV-like world.
>
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>    
>   
>
>
> -----Original Message-----
> From: Syed Ahmed [mailto:sahmed@cloudops.com]
> Sent: 19 September 2016 17:07
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Hey Guys,
>
> Will and I had a discussion in the morning on around VyOS and I have an
> idea that could work, here me out.
>
> The way Cloudstack currently communicates with VR is via the Hypervisor
> (let's assume KVM and XenServer for now). The management server sends a
> command as a JSON to the hypervisor which proxies them to the VR and they
> get executed there. Now, what we could do is add a proxy on the hypervisor
> which translates the management server JSON to VyOS commands (and
> vice-versa). There is a VyOS API lib in python which we clould use. Now
> this would require 0 changes on the Cloudstack core networking side. There
> may be a few minor changes required to push this proxy on the hypervisor
> but other than that there is nothing major. Now in the mean time, the VyOS
> guys can work on their v2.0 and when they have a stable enough API we can
> make it a first class citizen in Cloudstack.
>
> This approach would not work for VMWare as in VMWare the management server
> directly talks to the VR. However, this problem could be fixed by adding a
> middle VM which does the proxying to-and-from the VR. This would also
> improve the overall security of the system in VMWare as well.
>
> I am not too concerned about VMWare at the moment as most of us use either
> KVM or XenServer. What do you guys think of this approach?
>
> -Syed
>
> On Mon, Sep 19, 2016 at 11:42 AM, Stephan Seitz <
> s.seitz@secretresearchfacility.com> wrote:
>
>> Hi!
>>
>> Just to add my 2 cents to that thread:
>>
>> I'ld really like to see something like vyatta or pfsense integrated as
>> "standard" VR.
>>
>> We'd also talked internally about replacing the VR with some more
>> mature "appliance"-like router distro.
>>
>> pfsense e.g. comes AFAIK with no defined API but instead has a very
>> nice GUI.
>> How would this fit into the concept of configuring the VR via ACS?
>> Would parts of the GUI - like IP configuration and basic firewall rules
>>   - hidden or greyed?
>> Where would one save the configuration, VPN certificates and so on?
>>
>>
>> - Stephan
>>
>>
>> Am Sonntag, den 18.09.2016, 15:19 +0000 schrieb Marty Godsey:
>>> On this note I also mentioned pfsense earlier.
>>>
>>> www.pfsense.org
>>>
>>>
>>> Regards,
>>> Marty Godsey
>>>
>>> -----Original Message-----
>>> From: ilya [mailto:ilya.mailing.lists@gmail.com]
>>> Sent: Sunday, September 18, 2016 1:09 AM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [DISCUSS] Replacing the VR
>>>
>>> Our options become much better if we consider BSD based routers.
>>>
>>> Would that be on the table?
>>>
>>> https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributio
>>> ns
>>>
>>>
>>> On 9/16/16 12:04 PM, Will Stevens wrote:
>>>> Ya, your points are all valid Simon.  The lack of standard
>>>> libraries
>>>> to handle a lot of the details is a problem.  I don't think it is
>>>> an
>>>> unsolvable problem, but if we spend the time to do that, will we
>>>> have
>>>> something that will work for us for the next 5 years?  This may be
>>>> the
>>>> shortest path to getting us where we need to be for the time being.
>>>>
>>>> What is the best case scenario for the VR going forward which will
>>>> last us the next 5 years?  Maybe we just clean up what we have to
>>>> do a
>>>> major restructuring of the pieces and how they are
>>>> implemented.  We
>>>> need to keep in mind how maintainable this implementation is
>>>> because
>>>> that is going to be key going forward IMO.
>>>>
>>>>
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>> tw
>>>> @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sweller@ena.com>
>>>> wrote:
>>>>
>>>>> I think our other option is to take a real look at what it would
>>>>> take
>>>>> to fix the VR. In my opinion, a lot of the problems are related
>>>>> to
>>>>> the monolithic python code base and the fact nothing is actually
>>>>> separated.
>>>>>
>>>>> Secondly, the python scripts (and bash scripts) don't use any
>>>>> established libraries to complete tasks and instead shell out and
>>>>> run
>>>>> commands that are both hard to track and hard to parse on return.
>>>>>
>>>>>
>>>>> If we daemonized this, used a real api for Agent to VR
>>>>> communication,
>>>>> used common already existing libraries for the system service
>>>>> and
>>>>> network interactions and spent a bit of time separating out code
>>>>> into
>>>>> distinct modules, everything would behave a lot better.
>>>>>
>>>>>
>>>>> The pain and suffering is due to years and years of patches and
>>>>> constant shelling out to complete tasks in my opinion. If we
>>>>> spend
>>>>> time to rethink how we interact with the VR in general and we
>>>>> abstract the systems and networking stuff and use well known and
>>>>> stable libraries to do the work, the VR would be much easier to
>>>>> maintain.
>>>>>
>>>>>
>>>>> - Si
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ________________________________
>>>>> From: Marty Godsey <marty@gonsource.com>
>>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>
>>>>> So based upon this discussion would it be prudent to wait on
>>>>> VyOS
>>>>> 2.0? The current VR is giving us issues but would the time
>>>>> invested
>>>>> in another "solution" be wasted especially if by the time
>>>>> another
>>>>> option is chose, then coded, then tested, then implemented and
>>>>> right
>>>>> as that time happened to be when VyOS 2.0 is released.  Of course
>>>>> you
>>>>> said they are just in the scoping range so this could still be a
>>>>> year or more out.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Regards,
>>>>> Marty Godsey
>>>>> nSource Solutions
>>>>>
>>>>> -----Original Message-----
>>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
>>>>> On
>>>>> Behalf Of Will Stevens
>>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>>> To: dev@cloudstack.apache.org
>>>>> Cc: daniil@baturin.org
>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>> I just had a quick chat with a couple of the guys over on the
>>>>> VyOS chat.
>>>>> I have CC'ed one of them in case we have more licensing
>>>>> questions.
>>>>>
>>>>> So here is the status with the license "the code inherited from
>>>>> Vyatta and our modifications from it is GPLv2 (strict, not v2+).
>>>>> The
>>>>> config reading library is GPLv2 too, so anything that links to is
>>>>> is GPLv2.
>>>>> Some auxiliary components we made after the fork are more
>>>>> permissive,
>>>>> LGPLv2+ or MIT."
>>>>>
>>>>> They are currently in the process of scoping a redesign (VyOS
>>>>> 2.0),
>>>>> "we are planning a clean rewrite that will solve issues of the
>>>>> current config system".
>>>>> This will include the ability to configure via the API.
>>>>>
>>>>> If we have more questions for VyOS, they are very friendly and
>>>>> responsive, so we should be able to get answers.
>>>>>
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>> *|* tw
>>>>> @CloudOps_
>>>>>
>>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sahmed@cloudops.com>
>>>>> wrote:
>>>>>
>>>>>> I agree with Will Ilya. There are so many problems with the VR
>>>>>> right now.
>>>>>> Most of the outages we've had recently have somehow involved
>>>>>> the VR.
>>>>>> We set custom iptables rules on the VR which can and have
>>>>>> easily
>>>>>> gone
>>>>> wrong.
>>>>>> Openswan is broken, Strongswan replacement still needs to be
>>>>>> tested.
>>>>>> VVRP with redundant router still needs work, and not to mention
>>>>>> the
>>>>>> problems we will have when we introduce IPv6 into the whole
>>>>>> picture.
>>>>>>
>>>>>> I think the spirit of the discussion is to rely on a 3rd party
>>>>>> to do
>>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>>> orchestration. All the problems that I've described have
>>>>>> already
>>>>>> been solved in VyOS. We also get the advantage of a potential
>>>>>> wider
>>>>>> community to fix and maintain the VR and given our current
>>>>>> development velocity, it think it totally makes sense to look
>>>>>> for a 3rd party option.
>>>>>>
>>>>>> -Syed
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
>>>>>> <wstevens@cloudops.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The VR has been biting us far too often recently, which is
>>>>>>> why we
>>>>>>> have started looking into alternative implementations.
>>>>>>>
>>>>>>> One of the things that is nice about potentially using the
>>>>>>> VyOS is
>>>>>>> that
>>>>>> it
>>>>>>> is based on Debian, so we should be able to run the other
>>>>>>> services
>>>>>>> that
>>>>>> we
>>>>>>> currently have like the password server and userdata on the
>>>>>>> VyOS.
>>>>>>> This means we would not have to change our architecture
>>>>>>> initially
>>>>>>> and could focus on only replacing the networking paths.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>> cloudops.com *|*
>>>>>>> tw @CloudOps_
>>>>>>>
>>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nux@li.nux.ro> wrote:
>>>>>>>
>>>>>>>> The more this is discussed the more I think we should stick
>>>>>>>> with
>>>>>>>> our
>>>>>> VR.
>>>>>>>>
>>>>>>>> All these other options either seem unfinished or with
>>>>>>>> incompatible license.
>>>>>>>>
>>>>>>>> VyOS looks the most promising so far, it's a serious,
>>>>>>>> mature project.
>>>>>>>> Adopting it though means we'll have to microservice our way
>>>>>>>> out of
>>>>>>>> it
>>>>>>> with
>>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make
>>>>>>>> VyOS serve
>>>>>> those
>>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>
>>>>>>>> Nux!
>>>>>>>> www.nux.ro
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Will Stevens" <wstevens@cloudops.com>
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Ya, we would need to add a daemon for VPN as well.  Load
>>>>>>>>> balancing is another aspect which we will need to
>>>>>>>>> consider if we
>>>>> went this route.
>>>>>>>>> Something like https://traefik.io/ could potentially be a
>>>>>>>>> good
>>>>>>>>> fit
>>>>>> due
>>>>>>>> to
>>>>>>>>> its API driven configuration, but it may be more than
>>>>>>>>> what we need.
>>>>>>>>>
>>>>>>>>> We should probably try define which pieces make sense to
>>>>>>>>> be
>>>>>>>>> solved
>>>>>>>> together
>>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>>
>>>>>>>>> I think the network connectivity, routing and firewalling
>>>>>>>>> should
>>>>>>> probably
>>>>>>>>> all stay together since the majority of the tools we
>>>>>>>>> would
>>>>>> potentially
>>>>>>>> use
>>>>>>>>> would handle all of that together in a single
>>>>>>>>> implementation.
>>>>>>>>>
>>>>>>>>> The password server and userdata seems like a good option
>>>>>>>>> for
>>>>>>>>> being
>>>>>>>> broken
>>>>>>>>> out and handled independently (and probably rewritten
>>>>>>>>> completely
>>>>>> since
>>>>>>>> they
>>>>>>>>> currently have some issues).
>>>>>>>>>
>>>>>>>>> Load balancing is another that could warrant splitting
>>>>>>>>> out, but
>>>>>>>>> that depends on what direction we go and how we would be
>>>>>>>>> managing
>>>>> it.
>>>>>> DHCP
>>>>>>>> and
>>>>>>>>> DNS are others which could go either way.
>>>>>>>>>
>>>>>>>>> If we do split out services, I think we should
>>>>>>>>> consolidate as
>>>>>>>>> much as
>>>>>>> we
>>>>>>>>> can into each service we break out.  Ideally a network
>>>>>>>>> packet
>>>>>>>>> would
>>>>>>> never
>>>>>>>>> hit more than one, maybe two, services.  I don't think we
>>>>>>>>> should
>>>>>>>>> be splitting services 'just because', I think we need a
>>>>>>>>> valid
>>>>>>>>> case for splitting any service out because it adds
>>>>>>>>> complexity.
>>>>>>>>> Our project is already complex enough, we need to avoid
>>>>>>>>> adding
>>>>>>>>> complexity unless it
>>>>>> is
>>>>>>>>> really needed.
>>>>>>>>>
>>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>> cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sweller@e
>>>>>>>>> na.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I do agree with you that this probably isn't the right
>>>>>>>>>> place the
>>>>>>>> password
>>>>>>>>>> service and user data.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Having said that, after taking a cursory look at the
>>>>>>>>>> dev docs,
>>>>>>>>>> it
>>>>>>>> doesn't
>>>>>>>>>> seem that difficult to add new daemons:
>>>>>> https://opensnaproute.github.
>>>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>>
>>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>>> creating-new-component>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> They've definitely build it with a microservices
>>>>>>>>>> architecture in
>>>>>> mind,
>>>>>>>> so
>>>>>>>>>> each individual feature is abstracted into it's own
>>>>>>>>>> small daemon
>>>>>>>> process.
>>>>>>>>>> We could just create a daemon for the password server
>>>>>>>>>> and the
>>>>>> userdata
>>>>>>>>>> components if we really had to.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: williamstevens@gmail.com <williamstevens@gmail.co
>>>>>>>>>> m> on
>>>>>>>>>> behalf
>>>>>>> of
>>>>>>>>>> Will Stevens <wstevens@cloudops.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> A big part of why I know about it is because it is
>>>>>>>>>> written in Go.
>>>>>> :P
>>>>>>>>>>
>>>>>>>>>> Yes, it is definitely interesting for the routing and
>>>>>>>>>> traffic
>>>>>> handling
>>>>>>>>>> aspects of the VR.  We will likely have to rethink some
>>>>>>>>>> of the
>>>>>> pieces
>>>>>>> a
>>>>>>>>>> little bit like the password server and userdata if we
>>>>>>>>>> are to
>>>>>>>>>> adopt
>>>>>> a
>>>>>>>>>> different VR approach.  This is where I think some of
>>>>>>>>>> JohnB and
>>>>>>>> Chiradeep's
>>>>>>>>>> ideas make sense.  In many ways, it does not make sense
>>>>>>>>>> for the
>>>>>> device
>>>>>>>>>> handling routing and network traffic to also be
>>>>>>>>>> responsible for
>>>>>>>> passwords
>>>>>>>>>> and userdata.
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>> cloudops.com
>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sweller@
>>>>>>>>>> ena.com>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it.
>>>>>>>>>>> It looks
>>>>>> pretty
>>>>>>>>>> cool!
>>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>>
>>>>>>>>>>> To be honest, the ipsec feature could be added, or we
>>>>>>>>>>> could
>>>>>> offload
>>>>>>>> it to
>>>>>>>>>>> separate vm if we needed to. The fact it is so
>>>>>>>>>>> feature rich
>>>>>>>>>>> from a
>>>>>>>>>> routing
>>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Based on the roadmap, it looks like they plan to also
>>>>>>>>>>> support
>>>>>>>>>> capabilities
>>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the
>>>>>>>>>>> future. This
>>>>>>>>>>> will
>>>>>> be
>>>>>>>> huge
>>>>>>>>>>> for our carrier community that rely on these
>>>>>>>>>>> technologies to do
>>>>>>>> private
>>>>>>>>>>> gateway and inter-VPC interconnections today. We
>>>>>>>>>>> handle this
>>>>>>>>>>> stuff
>>>>>>> on
>>>>>>>> our
>>>>>>>>>>> ASRs right now with a vlan interconnect into the VR.
>>>>>>>>>>> Being able
>>>>>>>>>>> to
>>>>>>> do
>>>>>>>>>> MPLS
>>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It also seems to be written in GO (a language here at
>>>>>>>>>>> ENA we
>>>>>>>>>>> know
>>>>>>> very
>>>>>>>>>>> well).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Si
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ________________________________
>>>>>>>>>>> From: Will Stevens <williamstevens@gmail.com>
>>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Ya. I don't think it covers our whole use case, but
>>>>>>>>>>> what it
>>>>>>>>>>> does
>>>>>>>> cover is
>>>>>>>>>>> all api driven...
>>>>>>>>>>>
>>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <marty@gonsou
>>>>>>>>>>> rce.com>
>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense
>>>>>>>>>>>> since it was
>>>>>> not
>>>>>>>>>>>> intended to do IPSec.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems as though VyOS is starting to look like
>>>>>>>>>>>> the best
>>>>>> option.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>> ]
>>>>>>> On
>>>>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> Or we could go completely crazy and go with
>>>>>>>>>>>> something like
>>>>>>>> FlexSwitch
>>>>>>>>>>> from
>>>>>>>>>>>> SnapRoute
>>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>> cloudops.com
>>>>>>> *|*
>>>>>>>> tw
>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>> wstevens@cloudops.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I tend to agree with Syed and Marty.  I am not
>>>>>>>>>>>>> sure what
>>>>>>> problems
>>>>>>>> are
>>>>>>>>>>>>> solved by splitting up the function of the VR
>>>>>>>>>>>>> into a bunch of
>>>>>>>>>> separate
>>>>>>>>>>>>> services.  As Syed points out, the complexity
>>>>>>>>>>>>> added is
>>>>>>>> non-trivial.
>>>>>>>>>>>>> We now have to manage all the intercontainer
>>>>>>>>>>>>> networking as
>>>>>> well
>>>>>>> as
>>>>>>>>>> the
>>>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>>
>>>>>>>>>>>>> VyOS is interesting to me because it covers the
>>>>>>>>>>>>> majority of
>>>>>> our
>>>>>>>> use
>>>>>>>>>>>>> case with a single unified control plane.  It
>>>>>>>>>>>>> also has good
>>>>>>>> support
>>>>>>>>>>>>> for extending features we care about, like IPv6,
>>>>>>>>>>>>> VXLAN, VRRP,
>>>>>>>>>>>>> transactions, etc...
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>
>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>> cloudops.com
>>>>>>>> *|*
>>>>>>>>>> tw
>>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>> sahmed@cloudops.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Agree with Marty, adding Docker containers to
>>>>>>>>>>>>>> the picture
>>>>>>>> although
>>>>>>>>>>>>>> can make the VR more flexible but the added
>>>>>>>>>>>>>> complexity is
>>>>>> just
>>>>>>>> not
>>>>>>>>>>>>>> worth it. Not to mention we would need to take
>>>>>>>>>>>>>> care of
>>>>>>> networking
>>>>>>>>>>>>>> each container manually and given that our
>>>>>>>>>>>>>> iptable rules are
>>>>>>> very
>>>>>>>>>>>>>> unstable at the moment I don't see a big value
>>>>>>>>>>>>>> add.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Vyos looks like a better solution to me. I know
>>>>>>>>>>>>>> that it does
>>>>>>> not
>>>>>>>>>>>>>> provide an api but it does fit the bill quite
>>>>>>>>>>>>>> well
>>>>>> otherwise. I
>>>>>>>>>>>>>> specially like the fact that it has a
>>>>>>>>>>>>>> transaction based
>>>>>>>>>>>>>> model
>>>>>>> and
>>>>>>>>>> you
>>>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>> marty@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Licensing aside, I think splitting the
>>>>>>>>>>>>>>> various functions
>>>>>> into
>>>>>>>>>>>>>>> containers is not a good route either. This
>>>>>>>>>>>>>>> will force
>>>>>> users
>>>>>>> to
>>>>>>>>>>>>>>> have to maintain
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> use containers and adds complexity to the
>>>>>>>>>>>>>>> networking
>>>>>> aspects
>>>>>>> of
>>>>>>>>>> ACS.
>>>>>>>>>>>>>>> Complexity decreases stability. Now I
>>>>>>>>>>>>>>> understand the
>>>>>> argument
>>>>>>>> that
>>>>>>>>>>>>>>> a monolithic approach also brings its own set
>>>>>>>>>>>>>>> of issues but
>>>>>>> it
>>>>>>>>>> also
>>>>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gma
>>>>>>>>>>>>>>> il.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit
>>>>>>>>>>>>>>> the needs
>>>>>>>>>>>>>>> of
>>>>>> the
>>>>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>>>>>   - it is AGPL licensed. Many enterprises will
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> touch
>>>>>>>> anything
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>>>   - the github repo shows rather infrequent
>>>>>>>>>>>>>>> updates.
>>>>>>>>>>>>>>> Quite
>>>>>>>> likely
>>>>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>>> CloudStack
>>>>>>>> community
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'd back John B's comments on disaggregating
>>>>>>>>>>>>>>> the VR.
>>>>>>>>>>>>>>> Split
>>>>>> it
>>>>>>>> into
>>>>>>>>>>>>>>> many docker containers
>>>>>>>>>>>>>>>   - password server
>>>>>>>>>>>>>>>   - userdata server
>>>>>>>>>>>>>>>   - DHCP / DNS
>>>>>>>>>>>>>>>   - s2s VPN
>>>>>>>>>>>>>>>   - RA VPN
>>>>>>>>>>>>>>>   - intra-VPC routing and ACL
>>>>>>>>>>>>>>>   - Port forwarding + NAT
>>>>>>>>>>>>>>>   - FW
>>>>>>>>>>>>>>>   - LB (public)
>>>>>>>>>>>>>>>   - LB (internal),
>>>>>>>>>>>>>>>   - secondary storage
>>>>>>>>>>>>>>>   - agent
>>>>>>>>>>>>>>> Glue them together with  docker compose files
>>>>>>>>>>>>>>> (one per
>>>>>>>>>>>>>>> use
>>>>>>>> case -
>>>>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker.
>>>>>>>>>>>>>>> You can
>>>>>>>>>>>>>>> test
>>>>>> each
>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>>>> components independently and fixing one bug
>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>> field
>>>>>> (say
>>>>>>>>>> DHCP)
>>>>>>>>>>>>>>> is hitless to the other components. You don't
>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run
>>>>>>>>>>>>>>> on
>>>>> baremetal.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>>>   - make the traffic traverse the containers
>>>>>>>>>>>>>>> that are
>>>>>>>>>>>>>>> needed
>>>>>>> to
>>>>>>>> be
>>>>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>>>   - bootstrap the router (how does it find its
>>>>>>>>>>>>>>> compose
>>>>> file?
>>>>>>>> where
>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>>>   - rethink the command and control of the VR
>>>>>>>>>>>>>>> functions. SSH
>>>>>>>> works,
>>>>>>>>>>>>>>> but something more declarative, idempotent
>>>>>>>>>>>>>>> should be
>>>>>>> explored.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As you do this, it becomes clearer which of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> functions
>>>>>> can
>>>>>>>> be
>>>>>>>>>>>>>>> substituted by for example CloudRouter.
>>>>>>>>>>>>>>> Command and
>>>>>>>>>>>>>>> Control
>>>>>>> of
>>>>>>>> the
>>>>>>>>>>>>>> docker
>>>>>>>>>>>>>>> containers can be moved out to another
>>>>>>>>>>>>>>> container. Etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty
>>>>>>>>>>>>>>> Godsey
>>>>>>>>>>>>>>> <marty@gonsource.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This one does look nice. My biggest concern
>>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>>> lack
>>>>>> of
>>>>>>>>>>>>>>>> VXLANs. It seems that any of the ones we
>>>>>>>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>> do not
>>>>>>>> have
>>>>>>>>>> an
>>>>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cloudrouter looks promising. These have
>>>>>>>>>>>>>>>> potential to
>>>>>>>>>>>>>>>> save
>>>>>>>> future
>>>>>>>>>>>>>>>> engineering effort for example on ipv6
>>>>>>>>>>>>>>>> routing, OSPF
>>>>> etc.
>>>>>>>>>>>>>>>> And the best part is they come with test
>>>>>>>>>>>>>>>> automation
>>>>>>>> framework.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>>> <jayapal.uradi@accelerite.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Instead of replacing the VR in first
>>>>>>>>>>>>>>>>> place we
>>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>> as provider. Once it is stable, network
>>>>>>>>>>>>>>>> offerings
>>>>>>>>>>>>>>>> (on
>>>>>>>> upgrade)
>>>>>>>>>>>>>>>> can be updated to use it and we can drop
>>>>>>>>>>>>>>>> the VR if
>>>>>>>>>>>>>>>> we
>>>>>> want
>>>>>>> at
>>>>>>>>>>>>>>>> that release
>>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> VR is stabilized over a period of time
>>>>>>>>>>>>>>>>> and some of
>>>>>>>>>>>>>>>>> them
>>>>>>> are
>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>> without issues.  When we replicate the ACS
>>>>>>>>>>>>>>>> VR
>>>>>>>>>>>>>>>> features in
>>>>>>> new
>>>>>>>>>>>>>>>> solution it takes some to find the missing
>>>>>>>>>>>>>>>> pieces
>>>>>>>>>>>>>>>> (hidden
>>>>>>>> bugs).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm
>>>>>>>>>>>>>>>>>> not too
>>>>>>>>>>>>>>>>>> keen
>>>>>> on
>>>>>>>> VyOS
>>>>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg
>>>>>>>>>>>>>>>>>> technology!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>>> ;
>>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden,
>>>>>>>>>>>>>>>> London  WC2N 4HSUK
>>>>>>>> @shapeblue
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>> From: "Will Stevens" <williamstevens@
>>>>>>>>>>>>>>>>>>> gmail.com>
>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016
>>>>>>>>>>>>>>>>>>> 21:20:11
>>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought
>>>>>>>>>>>>>>>>>>> experiment and
>>>>>>>>>>>>>>>>>>> should
>>>>>>> be
>>>>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>> Please weigh in with the good and bad
>>>>>>>>>>>>>>>>>>> of this
>>>>> idea...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> A couple of us have been discussing
>>>>>>>>>>>>>>>>>>> the idea of
>>>>>>>> potentially
>>>>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS
>>>>>>>>>>>>>>>>>>> [1] (Open
>>>>>>>>>>>>>>>>>>> Source
>>>>>>>> Vyatta
>>>>>>>>>>> VM).
>>>>>>>>>>>>>>>>>>> There may be a license issue because
>>>>>>>>>>>>>>>>>>> I think it
>>>>>>>>>>>>>>>>>>> is
>>>>>>>> licensed
>>>>>>>>>>>>>>>>>>> under GPL, but for the sake of
>>>>>>>>>>>>>>>>>>> discussion, let's
>>>>>> assume
>>>>>>>> we
>>>>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have spent some time recently with
>>>>>>>>>>>>>>>>>>> the VyOS
>>>>>>>>>>>>>>>>>>> and I
>>>>>>> have
>>>>>>>> to
>>>>>>>>>>>>>>>>>>> admit, I was pretty impressed.  It is
>>>>>>>>>>>>>>>>>>> simple and
>>>>>>>> intuitive
>>>>>>>>>>>>>>>>>>> and it gives you a lot more options
>>>>>>>>>>>>>>>>>>> for auditing
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>>>> - Clean up our current VR script
>>>>>>>>>>>>>>>>>>> spaghetti to a
>>>>>> simpler
>>>>>>>> more
>>>>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6
>>>>>>>>>>>>>>>>>>> support.
>>>>>>>>>>>>>>>>>>> - Handles VPN configuration via the
>>>>>>>>>>>>>>>>>>> same
>>>>>> configuration
>>>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>>>>> - VPN support through OpenVPN &
>>>>>>>>>>>>>>>>>>> StrongSwan.
>>>>>>>>>>>>>>>>>>> - Easily supports HA (redundant
>>>>>>>>>>>>>>>>>>> routers) through
>>>>>> VRRP.
>>>>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>>>>> - Transaction based changes to the VR
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> rollback
>>>>>> on
>>>>>>>>>> error.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Items that could be difficult to
>>>>>>>>>>>>>>>>>>> solve:
>>>>>>>>>>>>>>>>>>> - Userdata password reset workflow
>>>>>>>>>>>>>>>>>>> and
>>>>>> implementation.
>>>>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The VyOS is not the only option if we
>>>>>>>>>>>>>>>>>>> were to
>>>>>> consider
>>>>>>>> this
>>>>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>>> Another option, which I don't know as
>>>>>>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>>>>> license) [2] which is purely API
>>>>>>>>>>>>>>>>>>> driven.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Anyway, would love to hear your
>>>>>>>>>>>>>>>>>>> thoughts...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>>>> This e-mail may contain privileged and
>>>>>>>>>>>>>>>>> confidential
>>>>>>>> information
>>>>>>>>>>>>>>>>> which is
>>>>>>>>>>>>>>>> the property of Accelerite, a Persistent
>>>>>>>>>>>>>>>> Systems
>>>>>> business.
>>>>>>>> It is
>>>>>>>>>>>>>>>> intended only for the use of the individual
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> entity to
>>>>>>>> which
>>>>>>>>>> it
>>>>>>>>>>>>>>>> is addressed. If you are not the intended
>>>>>>>>>>>>>>>> recipient,
>>>>>>>>>>>>>>>> you
>>>>>>> are
>>>>>>>> not
>>>>>>>>>>>>>>>> authorized to read, retain, copy, print,
>>>>>>>>>>>>>>>> distribute
>>>>>>>>>>>>>>>> or
>>>>>> use
>>>>>>>> this
>>>>>>>>>>>>>>>> message. If you have received this
>>>>>>>>>>>>>>>> communication in
>>>>>> error,
>>>>>>>>>> please
>>>>>>>>>>>>>>>> notify the sender and delete all copies of
>>>>>>>>>>>>>>>> this
>>>>> message.
>>>>>>>>>>>>>>>> Accelerite, a Persistent Systems business
>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>> accept
>>>>>>> any
>>>>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>>>>> infected mails.
>>>>>>>>>>>>>>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message