cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Burwell <jburw...@basho.com>
Subject Re: ACS + Riak-CS, works?
Date Thu, 31 Oct 2013 17:30:30 GMT
Darren,

Since S3/Riak CS can receive the contents of a file out of order and re-assemble upon completion
of transfer of all parts, the underlying driver uses the TransferManager which automatically
breaks files up into multi-part uploads to increase parallelism to decrease the time required
to upload to the object store.  Therefore, multi-part uploads are not employed solely to upload
files larger than 5 GB. 

Have you tried to upload the file to Riak CS using s3cmd outside of CloudStack?

Thanks,
-John

On Oct 31, 2013, at 1:23 PM, Darren Shepherd <darren.s.shepherd@gmail.com> wrote:

> My template is less than 5gb and it still uses multipart.  I actually don't think this
is a CloudStack issue but more of an issue with Riak-CS.  I was just wondering if anybody
has specifically ran the setup or knows more about Riak-CS and can tell me whats up.  I've
been trying off and on for the last week to try to get this working with 4.2 but it just keeps
failing.  I think I'll just try ceph rados gateway if I can't get this working.
> 
> Darren
> 
> 
> On Wed, Oct 30, 2013 at 11:17 PM, Sanjeev Neelarapu <sanjeev.neelarapu@citrix.com>
wrote:
> Hi,
> 
> Cloudstack with Riak-CS works fine. If we register template more than 5GB we use multipart
upload to push the files to Riak. But this does not work for snapshots and volumes. This will
be fixed in CS4.2.1 . But for template registration it should work fine.
> 
> Thanks,
> Sanjeev
> 
> -----Original Message-----
> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> Sent: Thursday, October 31, 2013 11:18 AM
> To: dev@cloudstack.apache.org
> Subject: ACS + Riak-CS, works?
> 
> Has anybody gotten CloudStack and Riak-CS to work together.  No matter what I try when
it goes to download and save the template, after its posted a series of multi part uploads
it ends up sending a 403.  The access log looks like
> 
> 127.0.0.1 - - [31/Oct/2013:05:41:11 +0000] "PUT
> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=14
> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic OpenJDK_64-Bit_Server_VM/20.0-b12"
> 127.0.0.1 - - [31/Oct/2013:05:41:28 +0000] "PUT
> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=15
> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic OpenJDK_64-Bit_Server_VM/20.0-b12"
> 127.0.0.1 - - [31/Oct/2013:05:41:42 +0000] "PUT
> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=16
> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic OpenJDK_64-Bit_Server_VM/20.0-b12"
> 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT
> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=17
> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic OpenJDK_64-Bit_Server_VM/20.0-b12"
> 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT
> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=18
> HTTP/1.1" 403 311 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic OpenJDK_64-Bit_Server_VM/20.0-b12"
> 
> And then in the error log I always get
> 
> 2013-10-31 05:41:51.867 [error]
> <0.5117.0>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user record for
s3 failed. Reason: disconnected
> 
> I'm able to interact with Riak-CS with s3cmd just fine but its not using multipart. I
think I'm going to try to download a new s3cmd that does multipart and see if even that will
work.
> 
> Darren
> 


Mime
View raw message