cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Mikhailovsky <and...@arhont.com>
Subject Re: S3 Secondary Storage fails with large files
Date Thu, 31 Oct 2013 17:37:30 GMT


Yes, indeed, this is how i am trying to create the template. I do get the same error as I
get while trying to snapshot. 

Is there a workaround or do i need to wait for the 4.2.1 to come out? 

Thanks 

Andrei 

----- Original Message -----

From: "Min Chen" <min.chen@citrix.com> 
To: dev@cloudstack.apache.org, users@cloudstack.apache.org 
Sent: Wednesday, 30 October, 2013 5:39:15 PM 
Subject: Re: S3 Secondary Storage fails with large files 

So you are doing create template from volume, right? I think that it 
should go through the same code used in backup snapshot, so will have the 
same limit. Would you mind paste the stack trace you got when you got that 
error in this case? 

Thanks 
-min 

On 10/30/13 10:29 AM, "Andrei Mikhailovsky" <andrei@arhont.com> wrote: 

>Hi, 
> 
>I've just tried creating a template from a 10GB disk image file and it 
>has failed with the same error - EntityTooLarge. So, neither snapshots 
>nor templates over 5GB in size are working for me. Could someone confirm 
>if I am not the only one? 
> 
>Thanks 
> 
>Andrei 
>----- Original Message ----- 
> 
>From: "Min Chen" <min.chen@citrix.com> 
>To: dev@cloudstack.apache.org, users@cloudstack.apache.org 
>Sent: Wednesday, 30 October, 2013 4:40:48 PM 
>Subject: Re: S3 Secondary Storage fails with large files 
> 
>For 4.2.0, you should be able to register template > 5GB. But we are 
>aware 
>of an issue you mentioned here: backup snapshot > 5GB will fail since 
>that 
>part of code is still using S3 single part upload api. This bug has been 
>fixed in latest 4.2 branch, and will be available in 4.2.1 release. 
> 
>Thanks 
>-min 
> 
>On 10/30/13 9:18 AM, "Andrei Mikhailovsky" <andrei@arhont.com> wrote: 
> 
>>Hello Guys , 
>> 
>>I am using ACS 4.2.0 with KVM hypervisor. While trying to create 
>>templates and snapshots from the existing vms I am getting an error 
>>message / exception on the management logs: 
>> 
>>2013-10-30 13:29:07,264 DEBUG [storage.snapshot.SnapshotManagerImpl] 
>>(Job-Executor-12:job-41 = [ 85659739-df2e-4afe-b638-2d25e804063f ]) 
>>Failed to create snapshot 
>>com.cloud.utils.exception.CloudRuntimeException: Status Code: 400, AWS 
>>Service: Amazon S3, AWS Request ID: null, AWS Error Code: 
>>EntityTooLarge, 
>>AWS Error Message: null, S3 Extended Request ID: null 
>>at 
>>com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient. 
>>j 
>>ava:610) 
>>at 
>>com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:3 
>>1 
>>0) 
>>at 
>>com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:164) 
>>at 
>>com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2906) 
>> 
>>at 
>>com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:11 
>>2 
>>3) 
>>at 
>>com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:97 
>>9 
>>) 
>>at com.cloud.utils.S3Utils.putFile(S3Utils.java:143) 
>>at 
>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.copyToS3(KVMStorageP 
>>r 
>>ocessor.java:556) 
>>at 
>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.copyToObjectStore(KV 
>>M 
>>StorageProcessor.java:582) 
>>at 
>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.backupSnapshotForObj 
>>e 
>>ctStore(KVMStorageProcessor.java:622) 
>>at 
>>com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.backupSnapshot(KVMSt 
>>o 
>>rageProcessor.java:634) 
>>at 
>>com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(Sto 
>>r 
>>ageSubsystemCommandHandlerBase.java:84) 
>>at 
>>com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStora 
>>g 
>>eCommands(StorageSubsystemCommandHandlerBase.java:49) 
>>at 
>>com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest 
>>( 
>>LibvirtComputingResource.java:1286) 
>>at com.cloud.agent.Agent.processRequest(Agent.java:525) 
>>at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) 
>>at com.cloud.utils.nio.Task.run(Task.java:83) 
>>at 
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java 
>>: 
>>1146) 
>>at 
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav 
>>a 
>>:615) 
>>at java.lang.Thread.run(Thread.java:679) 
>> 
>>at 
>>org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot 
>>( 
>>SnapshotServiceImpl.java:280) 
>>at 
>>org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSn 
>>a 
>>pshot(XenserverSnapshotStrategy.java:138) 
>>at 
>>org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnap 
>>s 
>>hot(XenserverSnapshotStrategy.java:264) 
>>at 
>>com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManag 
>>e 
>>rImpl.java:1013) 
>>at 
>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Interceptor 
>>D 
>>ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) 
>>at 
>>org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(Volum 
>>e 
>>ServiceImpl.java:1307) 
>>at 
>>com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2 
>>7 
>>19) 
>>at 
>>org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute 
>>( 
>>CreateSnapshotCmd.java:170) 
>>at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) 
>>at 
>>com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) 
>>at 
>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
>>at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
>>at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
>>at 
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java 
>>: 
>>1146) 
>>at 
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav 
>>a 
>>:615) 
>>at java.lang.Thread.run(Thread.java:679) 
>>2013-10-30 13:29:07,270 DEBUG [storage.volume.VolumeServiceImpl] 
>>(Job-Executor-12:job-41 = [ 85659739-df2e-4afe-b638-2d25e804063f ]) Take 
>>snapshot: 41 failed 
>>com.cloud.utils.exception.CloudRuntimeException: Failed to create 
>>snapshot 
>>at 
>>com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManag 
>>e 
>>rImpl.java:1040) 
>>at 
>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Interceptor 
>>D 
>>ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) 
>>at 
>>org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(Volum 
>>e 
>>ServiceImpl.java:1307) 
>>at 
>>com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2 
>>7 
>>19) 
>>at 
>>org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute 
>>( 
>>CreateSnapshotCmd.java:170) 
>>at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) 
>>at 
>>com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) 
>>at 
>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
>>at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
>>at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
>>at 
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java 
>>: 
>>1146) 
>>at 
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav 
>>a 
>>:615) 
>>at java.lang.Thread.run(Thread.java:679) 
>>Caused by: com.cloud.utils.exception.CloudRuntimeException: Status Code: 
>>400, AWS Service: Amazon S3, AWS Request ID: null, AWS Error Code: 
>>EntityTooLarge, AWS Error Message: null, S3 Extended Request ID: null 
>> 
>> 
>>Does anyone know how to resolve this problem? I can upload files up to 
>>5GB in size. Anything over that fails. 
>> 
>>Thanks for help 
>> 
>>Andrei 
>> 
> 
> 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message