cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Stevens <wstev...@cloudops.com>
Subject Re: [REVIEW] OpenStack Swift as Object Storage Service
Date Fri, 14 Mar 2014 21:46:36 GMT
When I originally wrote this integration there was no way to solve the
Same-Origin issue.  I wrote this implementation 2 years ago and I just
ported it the the 4.4 code base recently to potentially make available to
the community.

Currently: If you want to have Swift available on its own host which is not
accessed through CloudStack, that is fine.  If you want to have it
available through the CloudStack UI as well, you would just have the load
balancer redirect to the Swift hostname rather than an IP.

It is looking like people want significant changes to this code before they
will be happy with it, so it may not be realistic for me to make it
available at this time.  I do not have significant development hours that I
can put towards rebuilding this entire integration, so this may have to get
tabled until I have free hours (unless someone wants to sponsor the
redevelopment).

ws


On Fri, Mar 14, 2014 at 3:39 PM, Edison Su <Edison.su@citrix.com> wrote:

>  What I am trying to say is the public domain name for CloudStack mgt
> server and cloud storage(either swift and S3) can be different. For
> example, the cloudstack mgt server can have a domain name, like
> your-company.com, which mapping to a pubic ip address @ 52.53.53.55,
> while cloud storage can have a domain name, like,
> cloud-storage.your-company.com, which mapping to public ip address @
> 52.53.53.54. With different domain for different service, each service
> can scale and control differently.
>
> The client(browser) can call different domains, as most modern browsers
> supporting Cross-origin resource sharing now.
>
> The workflow will like this:
>
> 1.       Swift javascript client calls cloudstack mgt server api(e.g.
> listswfitcmd) to get the public domain of swift service and also get auth
> info. In your particular case, both public domain of swift service and
> cloudstack mgt server are the same.
>
> 2.       Swift javascript client calls public domain of swift
> service(returned by above step 1) for all the swift related operations.
>
> If we support above procedure, then it should be easy to add the support
> for AWS S3, or any other public cloud storage.
>
>
>
>
>
> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
> Behalf Of *Will Stevens
> *Sent:* Thursday, March 13, 2014 7:16 PM
>
> *To:* Edison Su
> *Cc:* dev@cloudstack.apache.org; David Nalley <david@gnsa.us> (
> david@gnsa.us)
> *Subject:* Re: [REVIEW] OpenStack Swift as Object Storage Service
>
>
>
> Lets say that we have the following setup to help clarify how this all
> works.
>
>
>
> CloudStack Management server is on: 10.10.10.2:8080
>
> Swift Proxy servers are on: 10.10.10.3:8080, 10.10.10.4:8080,
> 10.10.10.5:8080
>
> We have a load balancer setup on: 52.53.54.55:80
>
> The load balancer is networked to both the Swift cluster and CloudStack.
>
>
>
> When you want to connect to CloudStack, you connect to: 52.53.54.55:80
>
> You can basically think of that as your CloudStack IP since the load
> balancer passes all CloudStack traffic that hits 52.53.54.55:80 directly
> to CloudStack (10.10.10.2:8080).
>
>
>
> To everyone concerned, the IP 52.53.54.55:80 is the CS server because it
> behaves like it.
>
>
>
> The only URLs that do not go directly to the CloudStack box are URLs that
> start with 'v1.0' (swift auth urls) and 'v1' (all other swift calls).
>
>
>
> In this case we have 3 proxy servers on Swift, so the load balancer will
> load balance across the different Swift proxy servers when the URL starts
> with 'v1.0' or 'v1'.
>
>
>
> If we did not do it this way and we setup and endpoint for Swift in
> CloudStack, then we would bottleneck all of the traffic to Swift at the
> CloudStack management server.  The fact that we have 3 proxy servers would
> be made irrelevant because all the traffic would be going through the
> CloudStack management server first.  This would dramatically reduce the
> performance of Swift and would have a dramatic impact on the resources
> required by the CloudStack management server.
>
>
>
> By splitting the traffic before it gets to either of the services, we can
> increase the performance of both of them.
>
>
>
> Is that clearer?
>
>
>
> Will
>
>
>
> On Thu, Mar 13, 2014 at 9:06 PM, Edison Su <Edison.su@citrix.com> wrote:
>
> The way to talk to swift endpoint, by using haproxy to redirect url
> request contains "/v1.0" to swift endpoint, so it assumes, the endpoint of
> swift is the same as cloudstack mgt server. Correct me, if I am wrong, I
> just try to figure how it works.
>
> But what if the swift endpoint is different from CloudStack mgt server?
> Take Aws console as an example, the aws console is console.aws.amazon.com,
> but the S3 endpoint is s3-console-us-standard.console.aws.amazon.com.
>
>
>
> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
> Behalf Of *Will Stevens
> *Sent:* Thursday, March 13, 2014 5:52 PM
> *To:* Edison Su
> *Cc:* dev@cloudstack.apache.org; David Nalley <david@gnsa.us> (
> david@gnsa.us)
> *Subject:* Re: [REVIEW] OpenStack Swift as Object Storage Service
>
>
>
> What are you referring to as the 'v1.0 url hack'?  Bypassing the
> cloudstack management server is an intentional design decision to improve
> performance.  The introduction of a load balancer in front of the two
> distinct services to maintain separation of their traffic is a good thing.
>
>
>
> I will review the UI tutorial you sent...
>
>
>
> I will check how much work it is to replace PLupload with the jQuery File
> Upload plugin.
>
>
>
> Cheers,
>
>
>
> Will
>
>
>
> On Thu, Mar 13, 2014 at 8:24 PM, Edison Su <Edison.su@citrix.com> wrote:
>
> One comment on the "/v1.0" url hack, seems changing mgt server is
> unavoidable, would it be easier to  add a new api in cloudstack, which can
> get the endpoint of swift server and the auth?
>
> And the UI change should be put under ui plugin:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/UI+Plugin+Tutorial,
> and also, as David mentioned, plupload is not compatible with Apache
> license, maybe something like:
> https://github.com/blueimp/jQuery-File-Upload, is  better to be used.
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message