cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Stevens <wstev...@cloudops.com>
Subject Re: [DISCUSS] Replacing the VR
Date Tue, 20 Sep 2016 19:59:59 GMT
​If the link wrapped, be sure to remove the space that was added by the
wrap.  The hash in the URL should be: ​5ef827605f884961b94881e928e7a250

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Tue, Sep 20, 2016 at 3:51 PM, Will Stevens <wstevens@cloudops.com> wrote:

> I am going to try to do a bit of a brain dump from me and my team in this
> post, and hopefully I can verbalize the ideas well enough that you can all
> follow along.  Please provide feedback, ideas and suggestions...
>
> *Preface:*
> *--------*
> I am currently approaching this problem with the following use cases in
> mind.
>
> - Stabilize/replace the VR for our production cloud deployments as well as
> give the community a free, open source and production grade alternative to
> the current VR.
>
> - I have been scoping the integration of the Fortinet Fortigate (hardware
> and virtual) into ACS to support the following features.
> -- All Isolated Guest Network traffic features (Firewall, PF, NAT,
> Client-to-Site VPN, etc)
> -- All VPC network traffic features (ACLs, PF, NAT, Client-to-Site and
> Site-to-Site VPN, Private Gateway)
> -- Public IPv6 support (NAT64 and NAT46) with IPv4 on the guest side.
>
> *The Setup:*
> *----------*
> I am not going to get into the details of the Fortinet integration details
> in this thread, you can expect a function spec from me soon describing my
> plans for this work.  I am currently waiting on hardware to validate my
> implementation strategy.
>
> In this post I will be focusing on the way ACS deals with the VR, network
> plugins and how this could be potentially adapted to give us more
> flexibility going forward.
>
> I have created a diagram [1] to outline the ideas presented in this post.
>
> So if you don't like my ideas here, you can simply use the VR as you
> currently do and you can basically ignore the dotted circles to the right
> of the 'existing VR' in the solid circle.
>
> The basic idea here, is when you create a network service offering and you
> select different Providers to handle different network capabilities, you
> can also specify a VM template for those providers (as well as service
> offering) which will be used for additional VMs which will be created to
> provision those services.  In this context, think of templates such as
> VyOS, FortiOS, or a VPX to understand the spirit of this approach.
>
> When a network is provisioned, the current VR will be deployed (ideally
> only offering services like; password server, userdata, DHCP, DNS), and in
> addition, any network provider templates which have been specified will
> also be provisioned.  Each 'network VM' will get a leg on both the public
> VLAN and the guest VLAN.  In the case of a VPC, the first tier will be a
> little special (I will get into this in a bit) and will not be able to be
> deleted until the VR(s) are all cleaned up and the network is completely
> destroyed.  This network is denoted as the 'Client MGMT' network in the
> diagram [1].
>
> You will notice that ONLY the 'existing VR' has network connectivity to
> the ACS MGMT server.  It may not necessarily be through a 169.x.x.x
> address, but that illustrates the point for now.  The reason for this is
> because we can not guarantee that a Guest User does may not gain access to
> these VRs, and in cases I will get into later, the Guest User may actually
> be given access to those VRs.  This does however mean that the ACS MGMT
> server does not have direct access to those VRs in order to configure
> them.  To solve this, the 'existing VR' would act as a proxy for the ACS
> Management server over the ACS MGMT network to pass configuration commands
> to the VRs to the right of the 'existing VR' depicted in the graphic.
>
> *The Details:*
> *------------*
> Now that you have a basic idea of the deployment structure I have in mind,
> let's discuss some of the finer details.
>
> - We should be able to relatively easily adapt existing external network
> plugins to be able to be provisioned with this mechanism (assuming they
> have a VM option).  The Palo Alto would be an example which I would likely
> use as a proof of concept for this adaptation.  It would be important that
> the external plugin could work as an external physical appliance OR as a
> dynamically provisioned VM appliance.
>
> - Enterprise grade network appliances which support VM deployment will no
> longer have to be pre-deployed and shared, but instead will be able to be
> dynamically provisioned on a network by network basis.
>
> - Existing external network appliance plugins targeting physical boxes
> will still work in concert with this approach without any need for
> modification.
>
> - We could add an 'Allow Guest Access' checkbox to the service provider
> section in the network service offering to give the Guest User access to
> their network appliance.  This is especially useful for an implementation
> like the FortiGate because it allows for the user to configure UTM features
> which ACS does not orchestrate but the appliance is capable of.  This also
> applies to devices like the NetScaler.  In the diagram [1] this is depicted
> by the dashed green lines between the devices and the 'Client MGMT'
> network.  So in the case of the NetScaler, you would get an NSIP on the
> Client MGMT network which would allow the Guest User to connect to that
> box.  I envision a URL pointing to the guest network IP with
> username/password returned for a network which a user can use to connect
> into that network appliance.
>
> - Similar to above, the Guest User will have much more visibility into the
> logging and access on the network.  I can't count how many times a customer
> has asked for an easy way to get the IPsec logs from the VR.
>
> I think this is enough to get the idea across.  I have worked through a
> bunch of examples on a white board with my team to try to work through a
> bunch of the use cases, edge cases and deployment scenarios, but I am sure
> there are situations we have not thought through yet.
>
> The current VR has been a constant pain for us and our service customers,
> so it is likely that we will be taking some action one way or another
> because this needs to be addressed.  I know there is a lot going on here,
> but hopefully I have been able to get our ideas across.
>
> Please ask questions or make suggestions.  Are there major pieces we are
> not taking into account?  Do you have use cases which would be simplified
> by this approach that I have not called out?
>
> Would love to hear your feedback...
>
> [1] https://objects-east.cloud.ca/v1/5ef827605f884961b94881e
> 928e7a250/swill/vr_network.png
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy <muralimmreddy@gmail.com>
> wrote:
>
>> Configuration management of network appliances particularly for Cloud and
>> NFV scenarios is still evolving area. Programmability is the not the
>> strength for even the most popular network operating systems like IOS,
>> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
>> with stock linux for the VR.
>>
>> VR was never integral/hardcoded option in CloudStack. Its always been a
>> plugin. CloudStack network orchestration is well abstracted and designed
>> with vision to compose a network with different set of providers for
>> different services. Yes that vision is not fully realised yet, and we don’t
>> have true service function chains. That would be different discussion topic.
>>
>> I tend to agree with Simon, as alternate/interim option we can take hard
>> look whats causing the problems with current VR integration. Personally, I
>> think it would be easier we take a cue from configuration managers and
>> network configuration solutions out there (for e.g promise theory based
>> Cisco ACI) move to more declarative model of expressing desired state of
>> network configuration. Infact current VR from 4.6, actually holds the
>> desired state per service basis, seems to be in that direction.
>>
>> It does make sense to evaluate new appliances which can provide rich
>> semantics (like programmable API, declarative configuration, versioning,
>> commit/rollback etc), but will need significant engineering effort and time
>> to stabilise. We may make same mistakes with integration of other appliance
>> as well, if we fully don’t understand the nature of the current problems
>> with CloudStack core and service provider interaction and current VR
>> integration.
>>
>>
>> On 16/09/16, 11:59 PM, "Simon Weller" <sweller@ena.com> wrote:
>>
>> >I think our other option is to take a real look at what it would take to
>> fix the VR. In my opinion, a lot of the problems are related to the
>> monolithic python code base and the fact nothing is actually separated.
>> >
>> >Secondly, the python scripts (and bash scripts) don't use any
>> established libraries to complete tasks and instead shell out and run
>> commands that are both hard to track and hard to parse on return.
>> >
>> >
>> >If we daemonized this, used a real api for Agent to VR communication,
>> used common already existing libraries for the system service and network
>> interactions and spent a bit of time separating out code into distinct
>> modules, everything would behave a lot better.
>> >
>> >
>> >The pain and suffering is due to years and years of patches and constant
>> shelling out to complete tasks in my opinion. If we spend time to rethink
>> how we interact with the VR in general and we abstract the systems and
>> networking stuff and use well known and stable libraries to do the work,
>> the VR would be much easier to maintain.
>> >
>> >
>> >- Si
>> >
>> >
>> >
>> >
>> >________________________________
>> >From: Marty Godsey <marty@gonsource.com>
>> >Sent: Friday, September 16, 2016 12:24 PM
>> >To: dev@cloudstack.apache.org
>> >Subject: RE: [DISCUSS] Replacing the VR
>> >
>> >So based upon this discussion would it be prudent to wait on VyOS 2.0?
>> The current VR is giving us issues but would the time invested in another
>> "solution" be wasted especially if by the time another option is chose,
>> then coded, then tested, then implemented and right as that time happened
>> to be when VyOS 2.0 is released.  Of course you said they are just in the
>> scoping range so this could still be a year or more out.
>> >
>> >Thoughts?
>> >
>> >Regards,
>> >Marty Godsey
>> >nSource Solutions
>> >
>> >-----Original Message-----
>> >From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>> >Sent: Friday, September 16, 2016 10:31 AM
>> >To: dev@cloudstack.apache.org
>> >Cc: daniil@baturin.org
>> >Subject: Re: [DISCUSS] Replacing the VR
>> >
>> >I just had a quick chat with a couple of the guys over on the VyOS chat.
>> >I have CC'ed one of them in case we have more licensing questions.
>> >
>> >So here is the status with the license "the code inherited from Vyatta
>> and our modifications from it is GPLv2 (strict, not v2+). The config
>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>> >Some auxiliary components we made after the fork are more permissive,
>> >LGPLv2+ or MIT."
>> >
>> >They are currently in the process of scoping a redesign (VyOS 2.0), "we
>> are planning a clean rewrite that will solve issues of the current config
>> system".
>> >This will include the ability to configure via the API.
>> >
>> >If we have more questions for VyOS, they are very friendly and
>> responsive, so we should be able to get answers.
>> >
>> >*Will STEVENS*
>> >Lead Developer
>> >
>> >*CloudOps* *| *Cloud Solutions Experts
>> >420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>> >
>> >On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sahmed@cloudops.com> wrote:
>> >
>> >> I agree with Will Ilya. There are so many problems with the VR right
>> now.
>> >> Most of the outages we've had recently have somehow involved the VR.
>> >> We set custom iptables rules on the VR which can and have easily gone
>> wrong.
>> >> Openswan is broken, Strongswan replacement still needs to be tested.
>> >> VVRP with redundant router still needs work, and not to mention the
>> >> problems we will have when we introduce IPv6 into the whole picture.
>> >>
>> >> I think the spirit of the discussion is to rely on a 3rd party to do
>> >> the networking for us (eg VyOS) and have us handle just the
>> >> orchestration. All the problems that I've described have already been
>> >> solved in VyOS. We also get the advantage of a potential wider
>> >> community to fix and maintain the VR and given our current development
>> >> velocity, it think it totally makes sense to look for a 3rd party
>> option.
>> >>
>> >> -Syed
>> >>
>> >>
>> >> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <wstevens@cloudops.com>
>> >> wrote:
>> >>
>> >> > The VR has been biting us far too often recently, which is why we
>> >> > have started looking into alternative implementations.
>> >> >
>> >> > One of the things that is nice about potentially using the VyOS is
>> >> > that
>> >> it
>> >> > is based on Debian, so we should be able to run the other services
>> >> > that
>> >> we
>> >> > currently have like the password server and userdata on the VyOS.
>> >> > This means we would not have to change our architecture initially
>> >> > and could focus on only replacing the networking paths.
>> >> >
>> >> > *Will STEVENS*
>> >> > Lead Developer
>> >> >
>> >> > *CloudOps* *| *Cloud Solutions Experts
>> >> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> >> > tw @CloudOps_
>> >> >
>> >> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nux@li.nux.ro> wrote:
>> >> >
>> >> > > The more this is discussed the more I think we should stick with
>> >> > > our
>> >> VR.
>> >> > >
>> >> > > All these other options either seem unfinished or with
>> >> > > incompatible license.
>> >> > >
>> >> > > VyOS looks the most promising so far, it's a serious, mature
>> project.
>> >> > > Adopting it though means we'll have to microservice our way out of
>> >> > > it
>> >> > with
>> >> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>> >> those
>> >> > > too. Imho this adds complexity we should void.
>> >> > >
>> >> > > --
>> >> > > Sent from the Delta quadrant using Borg technology!
>> >> > >
>> >> > > Nux!
>> >> > > www.nux.ro
>> >> > >
>> >> > > ----- Original Message -----
>> >> > > > From: "Will Stevens" <wstevens@cloudops.com>
>> >> > > > To: dev@cloudstack.apache.org
>> >> > > > Sent: Thursday, 15 September, 2016 17:21:28
>> >> > > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > >
>> >> > > > Ya, we would need to add a daemon for VPN as well.  Load
>> >> > > > balancing is another aspect which we will need to consider if we
>> went this route.
>> >> > > > Something like https://traefik.io/ could potentially be a good
>> >> > > > fit
>> >> due
>> >> > > to
>> >> > > > its API driven configuration, but it may be more than what we
>> need.
>> >> > > >
>> >> > > > We should probably try define which pieces make sense to be
>> >> > > > solved
>> >> > > together
>> >> > > > and which pieces would be best suited to be broken out.
>> >> > > >
>> >> > > > I think the network connectivity, routing and firewalling should
>> >> > probably
>> >> > > > all stay together since the majority of the tools we would
>> >> potentially
>> >> > > use
>> >> > > > would handle all of that together in a single implementation.
>> >> > > >
>> >> > > > The password server and userdata seems like a good option for
>> >> > > > being
>> >> > > broken
>> >> > > > out and handled independently (and probably rewritten completely
>> >> since
>> >> > > they
>> >> > > > currently have some issues).
>> >> > > >
>> >> > > > Load balancing is another that could warrant splitting out, but
>> >> > > > that depends on what direction we go and how we would be
>> managing it.
>> >> DHCP
>> >> > > and
>> >> > > > DNS are others which could go either way.
>> >> > > >
>> >> > > > If we do split out services, I think we should consolidate as
>> >> > > > much as
>> >> > we
>> >> > > > can into each service we break out.  Ideally a network packet
>> >> > > > would
>> >> > never
>> >> > > > hit more than one, maybe two, services.  I don't think we should
>> >> > > > be splitting services 'just because', I think we need a valid
>> >> > > > case for splitting any service out because it adds complexity.
>> >> > > > Our project is already complex enough, we need to avoid adding
>> >> > > > complexity unless it
>> >> is
>> >> > > > really needed.
>> >> > > >
>> >> > > > Some more of my thoughts on this anyway...
>> >> > > >
>> >> > > > *Will STEVENS*
>> >> > > > Lead Developer
>> >> > > >
>> >> > > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> >> > > > *|* tw @CloudOps_
>> >> > > >
>> >> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sweller@ena.com>
>> >> > wrote:
>> >> > > >
>> >> > > >> I do agree with you that this probably isn't the right place
>> >> > > >> the
>> >> > > password
>> >> > > >> service and user data.
>> >> > > >>
>> >> > > >>
>> >> > > >> Having said that, after taking a cursory look at the dev docs,
>> >> > > >> it
>> >> > > doesn't
>> >> > > >> seem that difficult to add new daemons:
>> >> https://opensnaproute.github.
>> >> > > >> io/docs/developer.html#creating-new-component
>> >> > > >>
>> >> > > >> <https://opensnaproute.github.io/docs/developer.html#
>> >> > > >> creating-new-component>
>> >> > > >>
>> >> > > >>
>> >> > > >> They've definitely build it with a microservices architecture
>> >> > > >> in
>> >> mind,
>> >> > > so
>> >> > > >> each individual feature is abstracted into it's own small
>> >> > > >> daemon
>> >> > > process.
>> >> > > >> We could just create a daemon for the password server and the
>> >> userdata
>> >> > > >> components if we really had to.
>> >> > > >>
>> >> > > >>
>> >> > > >> - Si
>> >> > > >>
>> >> > > >>
>> >> > > >> ________________________________
>> >> > > >> From: williamstevens@gmail.com <williamstevens@gmail.com> on
>> >> > > >> behalf
>> >> > of
>> >> > > >> Will Stevens <wstevens@cloudops.com>
>> >> > > >> Sent: Thursday, September 15, 2016 9:17 AM
>> >> > > >> To: dev@cloudstack.apache.org
>> >> > > >> Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >>
>> >> > > >> A big part of why I know about it is because it is written in
>> Go.
>> >> :P
>> >> > > >>
>> >> > > >> Yes, it is definitely interesting for the routing and traffic
>> >> handling
>> >> > > >> aspects of the VR.  We will likely have to rethink some of the
>> >> pieces
>> >> > a
>> >> > > >> little bit like the password server and userdata if we are to
>> >> > > >> adopt
>> >> a
>> >> > > >> different VR approach.  This is where I think some of JohnB and
>> >> > > Chiradeep's
>> >> > > >> ideas make sense.  In many ways, it does not make sense for the
>> >> device
>> >> > > >> handling routing and network traffic to also be responsible for
>> >> > > passwords
>> >> > > >> and userdata.
>> >> > > >>
>> >> > > >> *Will STEVENS*
>> >> > > >> Lead Developer
>> >> > > >>
>> >> > > >> *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> >> > > >> *|* tw @CloudOps_
>> >> > > >>
>> >> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sweller@ena.com>
>> >> > wrote:
>> >> > > >>
>> >> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
>> >> pretty
>> >> > > >> cool!
>> >> > > >> > It even supports ONIE install.
>> >> > > >> >
>> >> > > >> > To be honest, the ipsec feature could be added, or we could
>> >> offload
>> >> > > it to
>> >> > > >> > separate vm if we needed to. The fact it is so feature rich
>> >> > > >> > from a
>> >> > > >> routing
>> >> > > >> > perspective (and all API driven) is really nice.
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > Based on the roadmap, it looks like they plan to also support
>> >> > > >> capabilities
>> >> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>> >> > > >> > will
>> >> be
>> >> > > huge
>> >> > > >> > for our carrier community that rely on these technologies to
>> >> > > >> > do
>> >> > > private
>> >> > > >> > gateway and inter-VPC interconnections today. We handle this
>> >> > > >> > stuff
>> >> > on
>> >> > > our
>> >> > > >> > ASRs right now with a vlan interconnect into the VR. Being
>> >> > > >> > able to
>> >> > do
>> >> > > >> MPLS
>> >> > > >> > all the way to the VR would be awesome.
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > It also seems to be written in GO (a language here at ENA we
>> >> > > >> > know
>> >> > very
>> >> > > >> > well).
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > - Si
>> >> > > >> >
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > ________________________________
>> >> > > >> > From: Will Stevens <williamstevens@gmail.com>
>> >> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
>> >> > > >> > To: dev@cloudstack.apache.org
>> >> > > >> > Subject: RE: [DISCUSS] Replacing the VR
>> >> > > >> >
>> >> > > >> > Ya. I don't think it covers our whole use case, but what it
>> >> > > >> > does
>> >> > > cover is
>> >> > > >> > all api driven...
>> >> > > >> >
>> >> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <marty@gonsource.com>
>> >> > wrote:
>> >> > > >> >
>> >> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
>> >> > > >> > > was
>> >> not
>> >> > > >> > > intended to do IPSec.
>> >> > > >> > >
>> >> > > >> > > It seems as though VyOS is starting to look like the best
>> >> option.
>> >> > > >> > >
>> >> > > >> > > Regards,
>> >> > > >> > > Marty Godsey
>> >> > > >> > > nSource Solutions
>> >> > > >> > >
>> >> > > >> > > -----Original Message-----
>> >> > > >> > > From: williamstevens@gmail.com
>> >> > > >> > > [mailto:williamstevens@gmail.com
>> >> ]
>> >> > On
>> >> > > >> > > Behalf Of Will Stevens
>> >> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
>> >> > > >> > > To: dev@cloudstack.apache.org
>> >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > >
>> >> > > >> > > Or we could go completely crazy and go with something like
>> >> > > FlexSwitch
>> >> > > >> > from
>> >> > > >> > > SnapRoute
>> >> > > >> > > - http://www.snaproute.com/
>> >> > > >> > > - https://opensnaproute.github.io/docs/apis.html
>> >> > > >> > >
>> >> > > >> > > *Will STEVENS*
>> >> > > >> > > Lead Developer
>> >> > > >> > >
>> >> > > >> > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> >> > > >> > > cloudops.com
>> >> > *|*
>> >> > > tw
>> >> > > >> > > @CloudOps_
>> >> > > >> > >
>> >> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>> >> > > wstevens@cloudops.com>
>> >> > > >> > > wrote:
>> >> > > >> > >
>> >> > > >> > > > I tend to agree with Syed and Marty.  I am not sure what
>> >> > problems
>> >> > > are
>> >> > > >> > > > solved by splitting up the function of the VR into a
>> >> > > >> > > > bunch of
>> >> > > >> separate
>> >> > > >> > > > services.  As Syed points out, the complexity added is
>> >> > > non-trivial.
>> >> > > >> > > > We now have to manage all the intercontainer networking
>> >> > > >> > > > as
>> >> well
>> >> > as
>> >> > > >> the
>> >> > > >> > > > orchestrated ACS networking.
>> >> > > >> > > >
>> >> > > >> > > > VyOS is interesting to me because it covers the majority
>> >> > > >> > > > of
>> >> our
>> >> > > use
>> >> > > >> > > > case with a single unified control plane.  It also has
>> >> > > >> > > > good
>> >> > > support
>> >> > > >> > > > for extending features we care about, like IPv6, VXLAN,
>> >> > > >> > > > VRRP, transactions, etc...
>> >> > > >> > > >
>> >> > > >> > > > *Will STEVENS*
>> >> > > >> > > > Lead Developer
>> >> > > >> > > >
>> >> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> >> cloudops.com
>> >> > > *|*
>> >> > > >> tw
>> >> > > >> > > > @CloudOps_
>> >> > > >> > > >
>> >> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>> >> > sahmed@cloudops.com>
>> >> > > >> > wrote:
>> >> > > >> > > >
>> >> > > >> > > >> Agree with Marty, adding Docker containers to the
>> >> > > >> > > >> picture
>> >> > > although
>> >> > > >> > > >> can make the VR more flexible but the added complexity
>> >> > > >> > > >> is
>> >> just
>> >> > > not
>> >> > > >> > > >> worth it. Not to mention we would need to take care of
>> >> > networking
>> >> > > >> > > >> each container manually and given that our iptable rules
>> >> > > >> > > >> are
>> >> > very
>> >> > > >> > > >> unstable at the moment I don't see a big value add.
>> >> > > >> > > >>
>> >> > > >> > > >> Vyos looks like a better solution to me. I know that it
>> >> > > >> > > >> does
>> >> > not
>> >> > > >> > > >> provide an api but it does fit the bill quite well
>> >> otherwise. I
>> >> > > >> > > >> specially like the fact that it has a transaction based
>> >> > > >> > > >> model
>> >> > and
>> >> > > >> you
>> >> > > >> > > >> can rollback changes if something goes wrong.
>> >> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>> >> > > marty@gonsource.com>
>> >> > > >> > > wrote:
>> >> > > >> > > >>
>> >> > > >> > > >> > Licensing aside, I think splitting the various
>> >> > > >> > > >> > functions
>> >> into
>> >> > > >> > > >> > containers is not a good route either. This will force
>> >> users
>> >> > to
>> >> > > >> > > >> > have to maintain
>> >> > > >> > > >> and
>> >> > > >> > > >> > use containers and adds complexity to the networking
>> >> aspects
>> >> > of
>> >> > > >> ACS.
>> >> > > >> > > >> > Complexity decreases stability. Now I understand the
>> >> argument
>> >> > > that
>> >> > > >> > > >> > a monolithic approach also brings its own set of
>> >> > > >> > > >> > issues but
>> >> > it
>> >> > > >> also
>> >> > > >> > > >> > simplifies it.
>> >> > > >> > > >> >
>> >> > > >> > > >> > Regards,
>> >> > > >> > > >> > Marty Godsey
>> >> > > >> > > >> > nSource Solutions
>> >> > > >> > > >> >
>> >> > > >> > > >> > -----Original Message-----
>> >> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> >> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
>> >> > > >> > > >> > To: dev@cloudstack.apache.org
>> >> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > > >> >
>> >> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
>> >> > > >> > > >> > of
>> >> the
>> >> > > >> > > >> > CloudStack project
>> >> > > >> > > >> >  - it is AGPL licensed. Many enterprises will not
>> >> > > >> > > >> > touch
>> >> > > anything
>> >> > > >> > > >> > that
>> >> > > >> > > >> has
>> >> > > >> > > >> > AGPL
>> >> > > >> > > >> >  - the github repo shows rather infrequent updates.
>> >> > > >> > > >> > Quite
>> >> > > likely
>> >> > > >> > > >> > they aren't considering the use cases of the
>> >> > > >> > > >> > CloudStack
>> >> > > community
>> >> > > >> > > >> >
>> >> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
>> >> > > >> > > >> > Split
>> >> it
>> >> > > into
>> >> > > >> > > >> > many docker containers
>> >> > > >> > > >> >  - password server
>> >> > > >> > > >> >  - userdata server
>> >> > > >> > > >> >  - DHCP / DNS
>> >> > > >> > > >> >  - s2s VPN
>> >> > > >> > > >> >  - RA VPN
>> >> > > >> > > >> >  - intra-VPC routing and ACL
>> >> > > >> > > >> >  - Port forwarding + NAT
>> >> > > >> > > >> >  - FW
>> >> > > >> > > >> >  - LB (public)
>> >> > > >> > > >> >  - LB (internal),
>> >> > > >> > > >> >  - secondary storage
>> >> > > >> > > >> >  - agent
>> >> > > >> > > >> > Glue them together with  docker compose files (one per
>> >> > > >> > > >> > use
>> >> > > case -
>> >> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
>> >> > > >> > > >> >
>> >> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
>> >> > > >> > > >> > test
>> >> each
>> >> > of
>> >> > > >> the
>> >> > > >> > > >> > components independently and fixing one bug in the
>> >> > > >> > > >> > field
>> >> (say
>> >> > > >> DHCP)
>> >> > > >> > > >> > is hitless to the other components. You don't need to
>> >> > > >> > > >> > build per-hypervisor VRs. You could even run on
>> baremetal.
>> >> > > >> > > >> >
>> >> > > >> > > >> > Along the way you need to figure out how to
>> >> > > >> > > >> >  - make the traffic traverse the containers that are
>> >> > > >> > > >> > needed
>> >> > to
>> >> > > be
>> >> > > >> > > >> > traversed (in most cases just 1)
>> >> > > >> > > >> >  - bootstrap the router (how does it find its compose
>> file?
>> >> > > where
>> >> > > >> > > >> > is the
>> >> > > >> > > >> > registry?)
>> >> > > >> > > >> >  - rethink the command and control of the VR
>> >> > > >> > > >> > functions. SSH
>> >> > > works,
>> >> > > >> > > >> > but something more declarative, idempotent should be
>> >> > explored.
>> >> > > >> > > >> >
>> >> > > >> > > >> > As you do this, it becomes clearer which of the
>> >> > > >> > > >> > functions
>> >> can
>> >> > > be
>> >> > > >> > > >> > substituted by for example CloudRouter. Command and
>> >> > > >> > > >> > Control
>> >> > of
>> >> > > the
>> >> > > >> > > >> docker
>> >> > > >> > > >> > containers can be moved out to another container. Etc.
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>> >> > > >> > > >> > <marty@gonsource.com>
>> >> > > >> > > >> > wrote:
>> >> > > >> > > >> >
>> >> > > >> > > >> > > This one does look nice. My biggest concern is the
>> >> > > >> > > >> > > lack
>> >> of
>> >> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
>> >> > > >> > > >> > > do not
>> >> > > have
>> >> > > >> an
>> >> > > >> > > >> > > API so we may be stuck at the SSH method.
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > Regards,
>> >> > > >> > > >> > > Marty Godsey
>> >> > > >> > > >> > > nSource Solutions
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > -----Original Message-----
>> >> > > >> > > >> > > From: Abhinandan Prateek
>> >> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
>> >> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> >> > > >> > > >> > > To: dev@cloudstack.apache.org
>> >> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > Cloudrouter looks promising. These have potential to
>> >> > > >> > > >> > > save
>> >> > > future
>> >> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF
>> etc.
>> >> > > >> > > >> > > And the best part is they come with test automation
>> >> > > framework.
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
>> >> > > >> > > >> > > <jayapal.uradi@accelerite.com>
>> >> > > >> > > >> > > wrote:
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > >Hi,
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >Instead of replacing the VR in first place we
>> >> > > >> > > >> > > >should add VyOS/cloudrouter
>> >> > > >> > > >> > > as provider. Once it is stable, network offerings
>> >> > > >> > > >> > > (on
>> >> > > upgrade)
>> >> > > >> > > >> > > can be updated to use it and we can drop the VR if
>> >> > > >> > > >> > > we
>> >> want
>> >> > at
>> >> > > >> > > >> > > that release
>> >> > > >> > > >> > onwards.
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >VR is stabilized over a period of time and some of
>> >> > > >> > > >> > > >them
>> >> > are
>> >> > > >> > > >> > > >running
>> >> > > >> > > >> > > without issues.  When we replicate the ACS VR
>> >> > > >> > > >> > > features in
>> >> > new
>> >> > > >> > > >> > > solution it takes some to find the missing pieces
>> >> > > >> > > >> > > (hidden
>> >> > > bugs).
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >Thanks,
>> >> > > >> > > >> > > >Jayapal
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >> nux@li.nux.ro> wrote:
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Hi,
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> I like the idea.
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
>> >> > > >> > > >> > > >> keen
>> >> on
>> >> > > VyOS
>> >> > > >> > > >> > > >> (it
>> >> > > >> > > >> > > doesn't have a proper http api etc).
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> --
>> >> > > >> > > >> > > >> Sent from the Delta quadrant using Borg
>> technology!
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Nux!
>> >> > > >> > > >> > > >> www.nux.ro
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > abhinandan.prateek@shapeblue.com
>> >> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
>> >> > > >> > > >> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> >> > > @shapeblue
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > ----- Original Message -----
>> >> > > >> > > >> > > >>> From: "Will Stevens" <williamstevens@gmail.com>
>> >> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
>> >> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> >> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
>> >> > > >> > > >> > > >>> should
>> >> > be
>> >> > > >> > > >> > > >>> treated as
>> >> > > >> > > >> > > such.
>> >> > > >> > > >> > > >>> Please weigh in with the good and bad of this
>> idea...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> A couple of us have been discussing the idea of
>> >> > > potentially
>> >> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
>> >> > > >> > > >> > > >>> Source
>> >> > > Vyatta
>> >> > > >> > VM).
>> >> > > >> > > >> > > >>> There may be a license issue because I think it
>> >> > > >> > > >> > > >>> is
>> >> > > licensed
>> >> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
>> >> assume
>> >> > > we
>> >> > > >> > > >> > > >>> can overcome any
>> >> > > >> > > >> > > license issues.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> I have spent some time recently with the VyOS
>> >> > > >> > > >> > > >>> and I
>> >> > have
>> >> > > to
>> >> > > >> > > >> > > >>> admit, I was pretty impressed.  It is simple and
>> >> > > intuitive
>> >> > > >> > > >> > > >>> and it gives you a lot more options for auditing
>> >> > > >> > > >> > > >>> the
>> >> > > >> > > configuration etc...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Items of potential interest:
>> >> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
>> >> simpler
>> >> > > more
>> >> > > >> > > >> > > >>> auditable configuration workflow.
>> >> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
>> >> > > >> > > >> > > >>> - Handles VPN configuration via the same
>> >> configuration
>> >> > > >> > > interface.
>> >> > > >> > > >> > > >>> - Support for OSPF & BGP.
>> >> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
>> >> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
>> >> VRRP.
>> >> > > >> > > >> > > >>> - VXLAN support.
>> >> > > >> > > >> > > >>> - Transaction based changes to the VR with
>> >> > > >> > > >> > > >>> rollback
>> >> on
>> >> > > >> error.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Items that could be difficult to solve:
>> >> > > >> > > >> > > >>> - Userdata password reset workflow and
>> >> implementation.
>> >> > > >> > > >> > > >>> - Upgrade process.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> The VyOS is not the only option if we were to
>> >> consider
>> >> > > this
>> >> > > >> > > >> approach.
>> >> > > >> > > >> > > >>> Another option, which I don't know as well,
>> >> > > >> > > >> > > >>> would be CloudRouter (AGPL
>> >> > > >> > > >> > > >>> license) [2] which is purely API driven.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Will
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
>> >> > > >> > > >> > > >>> https://cloudrouter.org/
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >DISCLAIMER
>> >> > > >> > > >> > > >==========
>> >> > > >> > > >> > > >This e-mail may contain privileged and confidential
>> >> > > information
>> >> > > >> > > >> > > >which is
>> >> > > >> > > >> > > the property of Accelerite, a Persistent Systems
>> >> business.
>> >> > > It is
>> >> > > >> > > >> > > intended only for the use of the individual or
>> >> > > >> > > >> > > entity to
>> >> > > which
>> >> > > >> it
>> >> > > >> > > >> > > is addressed. If you are not the intended recipient,
>> >> > > >> > > >> > > you
>> >> > are
>> >> > > not
>> >> > > >> > > >> > > authorized to read, retain, copy, print, distribute
>> >> > > >> > > >> > > or
>> >> use
>> >> > > this
>> >> > > >> > > >> > > message. If you have received this communication in
>> >> error,
>> >> > > >> please
>> >> > > >> > > >> > > notify the sender and delete all copies of this
>> message.
>> >> > > >> > > >> > > Accelerite, a Persistent Systems business does not
>> >> > > >> > > >> > > accept
>> >> > any
>> >> > > >> > > >> > > liability for virus
>> >> > > >> > > >> > infected mails.
>> >> > > >> > > >> > >
>> >> > > >> > > >> >
>> >> > > >> > > >>
>> >> > > >> > > >
>> >> > > >> > > >
>> >> > > >> > >
>> >> > > >> >
>> >> > >
>> >> >
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message