airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pierce, Marlon" <marpi...@iu.edu>
Subject Re: Jetstream VM creation through Airavata
Date Thu, 21 Apr 2016 00:09:45 GMT
Hi Emre,

The current case is the first: manage VMs to run jobs for Airavata, coupled with an Apache
Mesos-based scheduling of jobs within the VMs.

We have a Jetstream VM with the PGA and Airavata installed there also.  


Marlon

On 4/20/16, 6:34 PM, "Emre Brookes" <emre@biochem.uthscsa.edu> wrote:

>Or are you working on creating dynamic VM's to run jobs via Airavata ?
>both that and getting an Airavata server VM or docker version running 
>there would be of benefit.
>
>-e.
>
>
>Emre Brookes wrote:
>> Hi Suresh,
>>
>> Just FYI - Glad to hear you are getting Airavata easy to deploy on 
>> Jetstream.
>> I am currently working on getting GenApp easy to deploy on the 
>> standard Ubuntu & CentOS images there.
>> I should be ready in the next month or two to revisit Airavata with a 
>> focus on Jetstream  ...
>>
>> Cheers,
>> Emre
>>
>>
>> Suresh Marru wrote:
>>> Pankaj,
>>>
>>> We need to fix this NPE. Meanwhile, I just created a testKey on the 
>>> keystone. Can you try again?
>>>
>>> Suresh
>>>
>>>> On Apr 20, 2016, at 3:42 PM, Pankaj Saha <psaha4@binghamton.edu 
>>>> <mailto:psaha4@binghamton.edu>> wrote:
>>>>
>>>> I have just use maven to build the same project. here is thedetails
>>>> # Common properties
>>>> publicKeyFile=/home/psaha4/.ssh/id_rsa.pub
>>>>
>>>> # Jetstream properties
>>>> jetstream_imageId=448636b0-450a-4b49-9bfe-b3135fc443e8
>>>> jetstream_flavorId=3
>>>>
>>>> OS_PROJECT_DOMAIN_NAME=tacc
>>>> OS_USER_DOMAIN_NAME=tacc
>>>> OS_PROJECT_NAME=scigap
>>>> OS_TENANT_NAME=scigap
>>>> OS_USERNAME=scigap
>>>> OS_PASSWORD=<not exposing here>
>>>> OS_AUTH_URL=https://jblb.jetstream-cloud.org:35357/v3
>>>> OS_IDENTITY_API_VERSION=3
>>>>
>>>> # Network Properties
>>>> OS_FLOATING_IP_POOL=public
>>>> OS_SUBNET_NAME=airavata
>>>>
>>>> The test case error that I am getting for the Test case is al below:
>>>> -------------------------------------------------------
>>>>  T E S T S
>>>> -------------------------------------------------------
>>>> Running org.apache.airavata.cloud.test.CloudIntfTest
>>>> AuthenticationException{message=Unauthorized, status=401}
>>>> at 
>>>> org.openstack4j.core.transport.HttpExceptionHandler.mapException(HttpExceptionHandler.java:36)
>>>> at 
>>>> org.openstack4j.core.transport.HttpExceptionHandler.mapException(HttpExceptionHandler.java:23)
>>>> at 
>>>> org.openstack4j.openstack.internal.OSAuthenticator.authenticateV3(OSAuthenticator.java:161)
>>>> at 
>>>> org.openstack4j.openstack.internal.OSAuthenticator.invoke(OSAuthenticator.java:78)
>>>> at 
>>>> org.openstack4j.openstack.client.OSClientBuilder$ClientV3.authenticate(OSClientBuilder.java:163)
>>>> at 
>>>> org.openstack4j.openstack.client.OSClientBuilder$ClientV3.authenticate(OSClientBuilder.java:127)
>>>> at 
>>>> org.apache.airavata.cloud.openstack.OS4JClientProvider.getOSClientV3(OS4JClientProvider.java:86)
>>>> at 
>>>> org.apache.airavata.cloud.openstack.OS4JClientProvider.getOSClient(OS4JClientProvider.java:49)
>>>> at 
>>>> org.apache.airavata.cloud.intf.impl.OpenstackIntfImpl.<init>(OpenstackIntfImpl.java:78)
>>>> at 
>>>> org.apache.airavata.cloud.test.CloudIntfTest.jetstreamCreateDeleteServerTest(CloudIntfTest.java:77)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at 
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at 
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>>>> at 
>>>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>>>> at 
>>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>>>> java.lang.NullPointerException
>>>> at 
>>>> org.apache.airavata.cloud.intf.impl.OpenstackIntfImpl.getKeyPair(OpenstackIntfImpl.java:215)
>>>> at 
>>>> org.apache.airavata.cloud.test.CloudIntfTest.jetstreamCreateDeleteServerTest(CloudIntfTest.java:92)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at 
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at 
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>>>> at 
>>>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>>>> at 
>>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>>>> [main] ERROR org.apache.airavata.cloud.intf.impl.OpenstackIntfImpl - 
>>>> Failed to retrieve keypair: testKey
>>>> java.lang.NullPointerException
>>>> at 
>>>> org.apache.airavata.cloud.intf.impl.OpenstackIntfImpl.createKeyPair(OpenstackIntfImpl.java:197)
>>>> at 
>>>> org.apache.airavata.cloud.test.CloudIntfTest.jetstreamCreateDeleteServerTest(CloudIntfTest.java:94)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at 
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at 
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>>>> at 
>>>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>>>> at 
>>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>>>> [main] ERROR org.apache.airavata.cloud.intf.impl.OpenstackIntfImpl - 
>>>> Failed to create keypair: testKey
>>>> java.lang.NullPointerException
>>>> at 
>>>> org.apache.airavata.cloud.test.CloudIntfTest.jetstreamCreateDeleteServerTest(CloudIntfTest.java:97)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at 
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at 
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>>>> at 
>>>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>>>> at 
>>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>>>> at 
>>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>>>> at 
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
>>>> at 
>>>> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>>>> at 
>>>> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>>>> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
>>>> 0.913 sec <<< FAILURE! - in 
>>>> org.apache.airavata.cloud.test.CloudIntfTest
>>>> jetstreamCreateDeleteServerTest(org.apache.airavata.cloud.test.CloudIntfTest)

>>>>  Time elapsed: 0.908 sec  <<< FAILURE!
>>>> java.lang.AssertionError: null
>>>> at org.junit.Assert.fail(Assert.java:86)
>>>> at org.junit.Assert.fail(Assert.java:95)
>>>> at 
>>>> org.apache.airavata.cloud.test.CloudIntfTest.jetstreamCreateDeleteServerTest(CloudIntfTest.java:128)
>>>>
>>>>
>>>>
>>>> On Wed, Apr 20, 2016 at 3:36 PM, Suresh Marru <smarru@apache.org 
>>>> <mailto:smarru@apache.org>> wrote:
>>>>
>>>>     Thats nice. Few suggestions:
>>>>
>>>>     * Eclipse or IDEA are good to write code, but to build and run
>>>>     tests from maven, command line is the best option.
>>>>     * Please use the mailing lists as much as possible so rest of us
>>>>     can also contribute. If even you use hangouts, please take time
>>>>     to explain what the issue is so it engages every one.
>>>>
>>>>     I am able to run the test cases and am able to program
>>>>     against OpenstackIntfImpl just fine. Can you elaborate on what is
>>>>     the authentication issue you are running into?
>>>>
>>>>     Suresh
>>>>
>>>>>     On Apr 20, 2016, at 3:30 PM, Pankaj Saha <psaha4@binghamton.edu
>>>>>     <mailto:psaha4@binghamton.edu>> wrote:
>>>>>
>>>>>     Hi Suresh,
>>>>>
>>>>>     I just had a hangout session with Mangirish regarding the same.
>>>>>     It looks like an authentication issue with openstack which he is
>>>>>     aware of. He has explained me his code and I am going to retry
>>>>>     fixing the problem.
>>>>>
>>>>>     I did not use maven to build his project instead I have export
>>>>>     his project as eclipse (mvn eclipse:eclipse) and trying to run
>>>>>     the junit test.
>>>>>
>>>>>     Will update you once I find something new.
>>>>>
>>>>>     Thanks
>>>>>     Pankaj
>>>>>
>>>>>
>>>>>     On Wed, Apr 20, 2016 at 3:21 PM, Suresh Marru <smarru@apache.org
>>>>>     <mailto:smarru@apache.org>> wrote:
>>>>>
>>>>>         Hi Pankaj,
>>>>>
>>>>>         Can you please be more clear? What is failing? Did you try
>>>>>         to build it using maven within the cloud provisioning
>>>>>         directory itself? Did you change the public key file in the
>>>>>         test_data.properties?
>>>>>
>>>>>         Suresh
>>>>>
>>>>>>         On Apr 20, 2016, at 2:56 PM, Pankaj Saha
>>>>>>         <psaha4@binghamton.edu <mailto:psaha4@binghamton.edu>>
wrote:
>>>>>>
>>>>>>         I am able to get Mangirish's code in the develop branch.
>>>>>>         After making it an eclipse project, when I am trying to run
>>>>>>         it as JUnit test, it's failing.
>>>>>>         Need help to fix this, possibly some configuration changes
>>>>>>         in the properties file or similar.
>>>>>>         It will be great we can talk over a hangout session or at
>>>>>>         least a chat session to make it work.
>>>>>>
>>>>>>         On Wed, Apr 20, 2016 at 12:05 PM, Mangirish Wagle
>>>>>>         <vaglomangirish@gmail.com
>>>>>>         <mailto:vaglomangirish@gmail.com>> wrote:
>>>>>>
>>>>>>             Hi Pankaj,
>>>>>>
>>>>>>             You may find a sample test code for the module in this
>>>>>>             unit test file:-
>>>>>>
>>>>>> https://github.com/apache/airavata/blob/develop/modules/cloud/cloud-provisioning/src/test/java/org/apache/airavata/cloud/test/CloudIntfTest.java
>>>>>>
>>>>>>             Also, if you want to test run the code, you may please
>>>>>>             follow this quick README:-
>>>>>>
>>>>>> https://github.com/apache/airavata/blob/develop/modules/cloud/cloud-provisioning/README
>>>>>>
>>>>>>             Please let me know if you need some more info or code
>>>>>>             walkthrough. Sending you the jetstream openrc
>>>>>>             credentials separately.
>>>>>>
>>>>>>             Thanks.
>>>>>>
>>>>>>             Regards,
>>>>>>             Mangirish
>>>>>>
>>>>>>             On Wed, Apr 20, 2016 at 11:42 AM, Suresh Marru
>>>>>>             <smarru@apache.org <mailto:smarru@apache.org>>
wrote:
>>>>>>
>>>>>>                 Hi Pankaj,
>>>>>>
>>>>>>                 Please switch to ‘develop’ branch and look for
the
>>>>>>                 cloud provisioning module -
>>>>>> https://github.com/apache/airavata/tree/develop/modules/cloud/cloud-provisioning
>>>>>>
>>>>>>                 Suresh
>>>>>>
>>>>>>>                 On Apr 20, 2016, at 11:37 AM, Pankaj Saha
>>>>>>>                 <psaha4@binghamton.edu
>>>>>>>                 <mailto:psaha4@binghamton.edu>> wrote:
>>>>>>>
>>>>>>>                 Hello Mangirish,
>>>>>>>                 I have latest Airavata mater branch installed
in
>>>>>>>                 my local system. Can you please give us some
clue
>>>>>>>                 how to start creating the VMs through you
>>>>>>>                 application. Please specify where we can find
your
>>>>>>>                 corresponding java code. Is it through PGA website?
>>>>>>>                 Please share the required password to me in a
>>>>>>>                 separate email.
>>>>>>>
>>>>>>>                 Thanks
>>>>>>>                 Pankaj
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Mime
View raw message