whirr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Lv (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (WHIRR-733) Need to support a new option called whirr.user-thread when it comes to the Openstack
Date Fri, 05 Jul 2013 03:27:48 GMT

     [ https://issues.apache.org/jira/browse/WHIRR-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jeremy Lv updated WHIRR-733:
----------------------------

    Attachment: diff.patch
    
> Need to support a new option called whirr.user-thread when it comes to the Openstack
> ------------------------------------------------------------------------------------
>
>                 Key: WHIRR-733
>                 URL: https://issues.apache.org/jira/browse/WHIRR-733
>             Project: Whirr
>          Issue Type: New Feature
>          Components: core
>    Affects Versions: 0.8.2
>         Environment: Ubuntu
> Openstack
>            Reporter: Jeremy Lv
>             Fix For: 0.9.0
>
>         Attachments: diff.patch, swift_test.zip
>
>
> I found it will be blocked when create a virtual machine based on the Openstack(It is
because the whirr project can't remove the swift container after created the new instance)

> Here's my reproduced steps:
> First of all, you must configrate the Openstack environments including the openstack
swift
> 1>. Configurate my properties file called zookeeper.properties(In my environment,
I have put the file in the directory of "/home/Jeremy/zookeeper.properties")
> {code}
> whirr.cluster-name=myhadoopcluster
> #whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker
> whirr.instance-templates=1 zookeeper
> whirr.provider=openstack-nova
> whirr.location-id=RegionOne
> whirr.identity=admin:admin
> whirr.credential=hastexo
> whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726
> whirr.endpoint = http://10.167.157.86:5000/v2.0
> whirr.blobstore-provider=swift-keystone
> whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/
> whirr.blobstore-identity=service:swift
> whirr.blobstore-credential=hastexo
> #whirr.blobstore-cache-container=swift
> #whirr.blobstore-location-id=sdb1
> whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war
> jclouds.openstack-nova.auto-generate-keypairs = true
> #jclouds.openstack-nova.auto-create-floating-ips = true
> whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
> whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
> {code}
> 2>. Download the whirr-0.8.2.zip to my local platform(/home/Jeremy/whirr-0.8.2) and
unzip it
> 3>. cd /home/Jeremy/whirr-0.8.2 and execute the following command:
> ./bin/whirr launch-cluster --config ../zookeeper.properties
> 4>.You will found the swift blob container can be created successfully but can't be
removed automatically after create the new instance.(It will be blocked during the process
of launch-cluster and can't be canced).
> After a further investigation about the whirr project, I found it is because of the following
code:
> {code:title=BlobCache.java|borderStyle=solid}
>   static {
>     /* Ensure that all created containers are removed when the JVM stops */
>     Runtime.getRuntime().addShutdownHook(new Thread() {
>       @Override
>       public void run() {
>         BlobCache.dropAndCloseAll();
>       }
>     });
>   }
> {code} 
> It is because the default value of the user thread in jclouds side is "0", even we create
a new thread in the whirr side, we can't delete the container here because of the following
Exception
> {code}
> Exception in thread "Thread-0" java.util.concurrent.RejectedExecutionException: Task
[delegate=[method=SwiftKeystoneAsyncClient.listContainers, request=GET http://10.167.157.86:8080/v1/AUTH_60d91f3ee474439e8edbbf9f15f424c1/?format=json
HTTP/1.1], executionList=com.google.common.util.concurrent.ExecutionList@1b8633e] rejected
from java.util.concurrent.ThreadPoolExecutor@7b7e61[Terminated, pool size = 0, active threads
= 0, queued tasks = 0, completed tasks = 8]
> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2013)
> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
> 	at org.jclouds.concurrent.config.DescribingExecutorService.execute(DescribingExecutorService.java:105)
> 	at org.jclouds.concurrent.Futures$FutureListener.addListener(Futures.java:122)
> 	at org.jclouds.concurrent.Futures$ListenableFutureAdapter.addListener(Futures.java:153)
> 	at com.google.common.util.concurrent.Futures.transform(Futures.java:269)
> 	at com.google.common.util.concurrent.Futures.transform(Futures.java:380)
> 	at org.jclouds.concurrent.Futures.compose(Futures.java:270)
> 	at org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)
> 	at org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)
> 	at org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:254)
> 	at org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:148)
> 	at $Proxy70.listContainers(Unknown Source)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:170)
> 	at $Proxy71.listContainers(Unknown Source)
> 	at com.jclouds.test.JCloudsSwift.deleteConatiner(JCloudsSwift.java:63)
> 	at com.jclouds.test.JCloudsSwift.access$0(JCloudsSwift.java:60)
> 	at com.jclouds.test.JCloudsSwift$1.run(JCloudsSwift.java:34)
> {code}
> All in all, I have also write a sample related to the jcloud and found if we define the
PROPERTY_USER_THREADS in the jclouds side, it will be successful to remove the swift container
though the Runtime.getRuntime().addShutdownHook(new Thread(). If we didn't define the PROPERTY_USER_THREADS
in the jclouds side and it will be thrown the RejectedExecutionException in calling the Blobstore.deleteContainer
method because of the default user thread is "0"
> So I think we need to define a new option to support this situation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message