cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darren Shepherd <darren.s.sheph...@gmail.com>
Subject Re: ACS + Riak-CS, works?
Date Thu, 31 Oct 2013 17:44:19 GMT
Uploading without multipart works with s3cmd.  The s3cmd I was using didn't
have multipart support, so I'll download a newer one and try it tonight.

Would you happen to know what the below means?  I get a 200 response for
parts 1,2,3,4,etc and then randomly one part will fail with the below.

riak_cs_wm_common:maybe_create_user:223 Retrieval of user record for s3
failed. Reason: disconnected

Darren



On Thu, Oct 31, 2013 at 10:30 AM, John Burwell <jburwell@basho.com> wrote:

> Darren,
>
> Since S3/Riak CS can receive the contents of a file out of order and
> re-assemble upon completion of transfer of all parts, the underlying driver
> uses the TransferManager which automatically breaks files up into
> multi-part uploads to increase parallelism to decrease the time required to
> upload to the object store.  Therefore, multi-part uploads are not employed
> solely to upload files larger than 5 GB.
>
> Have you tried to upload the file to Riak CS using s3cmd outside of
> CloudStack?
>
> Thanks,
> -John
>
> On Oct 31, 2013, at 1:23 PM, Darren Shepherd <darren.s.shepherd@gmail.com>
> wrote:
>
> My template is less than 5gb and it still uses multipart.  I actually
> don't think this is a CloudStack issue but more of an issue with Riak-CS.
> I was just wondering if anybody has specifically ran the setup or knows
> more about Riak-CS and can tell me whats up.  I've been trying off and on
> for the last week to try to get this working with 4.2 but it just keeps
> failing.  I think I'll just try ceph rados gateway if I can't get this
> working.
>
> Darren
>
>
> On Wed, Oct 30, 2013 at 11:17 PM, Sanjeev Neelarapu <
> sanjeev.neelarapu@citrix.com> wrote:
>
>> Hi,
>>
>> Cloudstack with Riak-CS works fine. If we register template more than 5GB
>> we use multipart upload to push the files to Riak. But this does not work
>> for snapshots and volumes. This will be fixed in CS4.2.1 . But for template
>> registration it should work fine.
>>
>> Thanks,
>> Sanjeev
>>
>> -----Original Message-----
>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>> Sent: Thursday, October 31, 2013 11:18 AM
>> To: dev@cloudstack.apache.org
>> Subject: ACS + Riak-CS, works?
>>
>> Has anybody gotten CloudStack and Riak-CS to work together.  No matter
>> what I try when it goes to download and save the template, after its posted
>> a series of multi part uploads it ends up sending a 403.  The access log
>> looks like
>>
>> 127.0.0.1 - - [31/Oct/2013:05:41:11 +0000] "PUT
>>
>> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=14
>> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic
>> OpenJDK_64-Bit_Server_VM/20.0-b12"
>> 127.0.0.1 - - [31/Oct/2013:05:41:28 +0000] "PUT
>>
>> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=15
>> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic
>> OpenJDK_64-Bit_Server_VM/20.0-b12"
>> 127.0.0.1 - - [31/Oct/2013:05:41:42 +0000] "PUT
>>
>> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=16
>> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic
>> OpenJDK_64-Bit_Server_VM/20.0-b12"
>> 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT
>>
>> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=17
>> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic
>> OpenJDK_64-Bit_Server_VM/20.0-b12"
>> 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT
>>
>> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=18
>> HTTP/1.1" 403 311 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic
>> OpenJDK_64-Bit_Server_VM/20.0-b12"
>>
>> And then in the error log I always get
>>
>> 2013-10-31 05:41:51.867 [error]
>> <0.5117.0>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user
>> record for s3 failed. Reason: disconnected
>>
>> I'm able to interact with Riak-CS with s3cmd just fine but its not using
>> multipart. I think I'm going to try to download a new s3cmd that does
>> multipart and see if even that will work.
>>
>> Darren
>>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message