cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Min Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-2706) [Object_Store_Refactor]Registering template with more than 5GB is failing with S3 object store
Date Wed, 05 Jun 2013 21:59:21 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13676408#comment-13676408
] 

Min Chen commented on CLOUDSTACK-2706:
--------------------------------------

Fixed with this commit 4e404953ad72862d7054cdd03eb6bea95d5fc00c. Use TransferManager to handle
multi-part upload.
                
> [Object_Store_Refactor]Registering template with more than 5GB is failing with S3 object
store
> ----------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-2706
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2706
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.0
>         Environment: Build from object_store
>            Reporter: Sanjeev N
>            Assignee: Min Chen
>            Priority: Critical
>             Fix For: 4.2.0
>
>
> Registering template with more than 5GB is failing with S3 object store:
> 1.Bring up CS in advanced zone with S3 as the storage provider
> 2.Try to register template/iso with more than 5GB size.
> Result:
> ======
> Registering template with more than 5GB size failed.
> Log snippet from SSVM:
> 2013-05-28 07:03:44,733 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Request:Seq
5-1626603540:  { Cmd , MgmtId: 7332683579487, via: 5, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DownloadCommand":{"hvm":true,"description":"win8","maxDownloadSizeInBytes":53687091200,"id":204,"resourceType":"TEMPLATE","installPath":"template/tmpl/2/204/204-2-4b75a684-2865-317b-8076-0108dd7c7cd5","_store":{"com.cloud.agent.api.to.S3TO":{"id":2,"uuid":"8046defe-63ff-44f8-8fd6-51a8e0cd430a","accessKey":"GLSDOYWL06R-NTXGU_VM","secretKey":"c9eS5SzPLfYkd-BetHKWhd-eSoYPWHjF6-xo4Q==","endPoint":"10.147.29.56:8080","bucketName":"imagestore","httpsFlag":false,"created":"May
27, 2013 8:31:46 AM"}},"url":"http://10.147.28.7/templates/Windows8/Windows864.vhd","format":"VHD","accountId":2,"name":"204-2-4b75a684-2865-317b-8076-0108dd7c7cd5","wait":0}}]
}
> 2013-05-28 07:03:44,733 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Processing
command: org.apache.cloudstack.storage.command.DownloadCommand
> 2013-05-28 07:03:44,734 INFO  [storage.template.S3TemplateDownloader] (agentRequest-Handler-5:)
No credentials configured for host=10.147.28.7:80
> 2013-05-28 07:03:44,817 INFO  [storage.template.S3TemplateDownloader] (pool-1-thread-2:)
Starting download from http://10.147.28.7/templates/Windows8/Windows864.vhd to s3 bucket imagestore
remoteSize=9436754432 , max size=53687091200
> 2013-05-28 07:03:44,818 DEBUG [cloud.utils.S3Utils] (pool-1-thread-2:) Creating S3 client
with configuration: [protocol: http, connectionTimeOut: 50000, maxErrorRetry: 3, socketTimeout:
50000]
> 2013-05-28 07:03:44,819 DEBUG [cloud.utils.S3Utils] (pool-1-thread-2:) Setting the end
point for S3 client com.amazonaws.services.s3.AmazonS3Client@135605a to 10.147.29.56:8080.
> 2013-05-28 07:03:47,734 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Seq 5-1626603540:
 { Ans: , MgmtId: 7332683579487, via: 5, Ver: v1, Flags: 10, [{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"0649731c-ce29-44ec-bdbb-bd718b5d1a31","downloadPct":0,"errorString":"
","downloadStatus":"DOWNLOAD_ERROR","downloadPath":"template/tmpl/2/204/204-2-4b75a684-2865-317b-8076-0108dd7c7cd5/Windows864.vhd","installPath":"template/tmpl/2/204/204-2-4b75a684-2865-317b-8076-0108dd7c7cd5","templateSize":0,"templatePhySicalSize":0,"result":true,"details":"
","wait":0}}] }
> As per the review comments on the FS: the s3 implementation should use multi-part upload
to transfer files larger than 5GB in size. If not the implementation will be limited to storing
files no larger than 5GB in size.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message