cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Min Chen <min.c...@citrix.com>
Subject Re: S3 Secondary Storage fails with large files
Date Thu, 31 Oct 2013 18:09:30 GMT
Sorry, but you may have to wait for 4.2.1 to come out.

Thanks
-min

On 10/31/13 10:37 AM, "Andrei Mikhailovsky" <andrei@arhont.com> wrote:

>
>
>Yes, indeed, this is how i am trying to create the template. I do get the
>same error as I get while trying to snapshot.
>
>Is there a workaround or do i need to wait for the 4.2.1 to come out?
>
>Thanks 
>
>Andrei 
>
>----- Original Message -----
>
>From: "Min Chen" <min.chen@citrix.com>
>To: dev@cloudstack.apache.org, users@cloudstack.apache.org
>Sent: Wednesday, 30 October, 2013 5:39:15 PM
>Subject: Re: S3 Secondary Storage fails with large files
>
>So you are doing create template from volume, right? I think that it
>should go through the same code used in backup snapshot, so will have the
>same limit. Would you mind paste the stack trace you got when you got
>that 
>error in this case?
>
>Thanks 
>-min 
>
>On 10/30/13 10:29 AM, "Andrei Mikhailovsky" <andrei@arhont.com> wrote:
>
>>Hi, 
>> 
>>I've just tried creating a template from a 10GB disk image file and it
>>has failed with the same error - EntityTooLarge. So, neither snapshots
>>nor templates over 5GB in size are working for me. Could someone confirm
>>if I am not the only one?
>> 
>>Thanks 
>> 
>>Andrei 
>>----- Original Message -----
>> 
>>From: "Min Chen" <min.chen@citrix.com>
>>To: dev@cloudstack.apache.org, users@cloudstack.apache.org
>>Sent: Wednesday, 30 October, 2013 4:40:48 PM
>>Subject: Re: S3 Secondary Storage fails with large files
>> 
>>For 4.2.0, you should be able to register template > 5GB. But we are
>>aware 
>>of an issue you mentioned here: backup snapshot > 5GB will fail since
>>that 
>>part of code is still using S3 single part upload api. This bug has been
>>fixed in latest 4.2 branch, and will be available in 4.2.1 release.
>> 
>>Thanks 
>>-min 
>> 
>>On 10/30/13 9:18 AM, "Andrei Mikhailovsky" <andrei@arhont.com> wrote:
>> 
>>>Hello Guys , 
>>> 
>>>I am using ACS 4.2.0 with KVM hypervisor. While trying to create
>>>templates and snapshots from the existing vms I am getting an error
>>>message / exception on the management logs:
>>> 
>>>2013-10-30 13:29:07,264 DEBUG [storage.snapshot.SnapshotManagerImpl]
>>>(Job-Executor-12:job-41 = [ 85659739-df2e-4afe-b638-2d25e804063f ])
>>>Failed to create snapshot
>>>com.cloud.utils.exception.CloudRuntimeException: Status Code: 400, AWS
>>>Service: Amazon S3, AWS Request ID: null, AWS Error Code:
>>>EntityTooLarge, 
>>>AWS Error Message: null, S3 Extended Request ID: null
>>>at 
>>>com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient
>>>. 
>>>j 
>>>ava:610) 
>>>at 
>>>com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:
>>>3 
>>>1 
>>>0) 
>>>at 
>>>com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:164)
>>>at 
>>>com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2906
>>>) 
>>> 
>>>at 
>>>com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1
>>>1 
>>>2 
>>>3) 
>>>at 
>>>com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:9
>>>7 
>>>9 
>>>) 
>>>at com.cloud.utils.S3Utils.putFile(S3Utils.java:143)
>>>at 
>>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.copyToS3(KVMStorage
>>>P 
>>>r 
>>>ocessor.java:556)
>>>at 
>>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.copyToObjectStore(K
>>>V 
>>>M 
>>>StorageProcessor.java:582)
>>>at 
>>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.backupSnapshotForOb
>>>j 
>>>e 
>>>ctStore(KVMStorageProcessor.java:622)
>>>at 
>>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.backupSnapshot(KVMS
>>>t 
>>>o 
>>>rageProcessor.java:634)
>>>at 
>>>com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(St
>>>o 
>>>r 
>>>ageSubsystemCommandHandlerBase.java:84)
>>>at 
>>>com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStor
>>>a 
>>>g 
>>>eCommands(StorageSubsystemCommandHandlerBase.java:49)
>>>at 
>>>com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeReques
>>>t 
>>>( 
>>>LibvirtComputingResource.java:1286)
>>>at com.cloud.agent.Agent.processRequest(Agent.java:525)
>>>at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
>>>at com.cloud.utils.nio.Task.run(Task.java:83)
>>>at 
>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
>>>a 
>>>: 
>>>1146) 
>>>at 
>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
>>>v 
>>>a 
>>>:615) 
>>>at java.lang.Thread.run(Thread.java:679)
>>> 
>>>at 
>>>org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapsho
>>>t 
>>>( 
>>>SnapshotServiceImpl.java:280)
>>>at 
>>>org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupS
>>>n 
>>>a 
>>>pshot(XenserverSnapshotStrategy.java:138)
>>>at 
>>>org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSna
>>>p 
>>>s 
>>>hot(XenserverSnapshotStrategy.java:264)
>>>at 
>>>com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotMana
>>>g 
>>>e 
>>>rImpl.java:1013)
>>>at 
>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercepto
>>>r 
>>>D 
>>>ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>>at 
>>>org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(Volu
>>>m 
>>>e 
>>>ServiceImpl.java:1307)
>>>at 
>>>com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:
>>>2 
>>>7 
>>>19) 
>>>at 
>>>org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execut
>>>e 
>>>( 
>>>CreateSnapshotCmd.java:170)
>>>at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>>>at 
>>>com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>>>at 
>>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>>>at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>>at 
>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
>>>a 
>>>: 
>>>1146) 
>>>at 
>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
>>>v 
>>>a 
>>>:615) 
>>>at java.lang.Thread.run(Thread.java:679)
>>>2013-10-30 13:29:07,270 DEBUG [storage.volume.VolumeServiceImpl]
>>>(Job-Executor-12:job-41 = [ 85659739-df2e-4afe-b638-2d25e804063f ])
>>>Take 
>>>snapshot: 41 failed
>>>com.cloud.utils.exception.CloudRuntimeException: Failed to create
>>>snapshot 
>>>at 
>>>com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotMana
>>>g 
>>>e 
>>>rImpl.java:1040)
>>>at 
>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercepto
>>>r 
>>>D 
>>>ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>>at 
>>>org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(Volu
>>>m 
>>>e 
>>>ServiceImpl.java:1307)
>>>at 
>>>com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:
>>>2 
>>>7 
>>>19) 
>>>at 
>>>org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execut
>>>e 
>>>( 
>>>CreateSnapshotCmd.java:170)
>>>at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>>>at 
>>>com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>>>at 
>>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>>>at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>>at 
>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
>>>a 
>>>: 
>>>1146) 
>>>at 
>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
>>>v 
>>>a 
>>>:615) 
>>>at java.lang.Thread.run(Thread.java:679)
>>>Caused by: com.cloud.utils.exception.CloudRuntimeException: Status
>>>Code: 
>>>400, AWS Service: Amazon S3, AWS Request ID: null, AWS Error Code:
>>>EntityTooLarge, AWS Error Message: null, S3 Extended Request ID: null
>>> 
>>> 
>>>Does anyone know how to resolve this problem? I can upload files up to
>>>5GB in size. Anything over that fails.
>>> 
>>>Thanks for help 
>>> 
>>>Andrei 
>>> 
>> 
>> 
>
>


Mime
View raw message