whirr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (WHIRR-697) Apache Whirr Core tests fail when running 'mvn install'
Date Fri, 08 Feb 2013 20:07:12 GMT

    [ https://issues.apache.org/jira/browse/WHIRR-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574776#comment-13574776
] 

Eli Reisman commented on WHIRR-697:
-----------------------------------

This part of the ClusterSpec's TemplateBuilderSpec filter for ubuntu images on Amazon:
{code}osDescription(^(?!.(daily|testing)).*ubuntu-images.$),osVersion(10.04){code}
is not working to detect valid images any more (10.04 is getting old as an AMI image, maybe
thats it) but point being I am putting up a patch that seems to pass all the tests now on
'mvn install' that just removes the filter. I have found lately while working on recipes/services
that this filter is not needed to find valid Ubuntu 10.04 images on AMI us-east-1 or us-west-2
using only the osFamily == UBUNTU and osVersion == 10.04 filters. Will post the patch in a
moment...

One issue (already posted on another JIRA) is when you crank up Ubuntu version to a more recent
one, the "update-alternatives" command from 10.04 release is broken for selecting default
Java version. That will need to be fixed in Hadoop service code if we wish to update our "defaults"
to a newer (or not version-specific) Ubuntu, of which there are (and will always be) many
AMI's available.

This might be worth a larger discussion, as it seems in certain critical spots there are defaults
like this in Whirr that might bear some generalizing to avoid having to update recipes etc.
so often to keep up with cloud provider's ever-changing image selection? Whirr/jclouds seems
to be able to make good decisions abotu AMI's (at least on Amazon) based on wider filter settings.
                
> Apache Whirr Core tests fail when running 'mvn install'
> -------------------------------------------------------
>
>                 Key: WHIRR-697
>                 URL: https://issues.apache.org/jira/browse/WHIRR-697
>             Project: Whirr
>          Issue Type: Bug
>            Reporter: Eli Reisman
>
> When downloading a clean repo on my Mac OSX Mountain Lion laptop, I can run 'mvn install'
and the tests will always fail like so:
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running org.apache.whirr.actions.BootstrapClusterActionTest
> 2013-02-02 13:55:31,806 WARN  [org.apache.whirr.ClusterSpec] (main) Please use provider
"aws-ec2" instead of "ec2"
> 2013-02-02 13:55:32,442 INFO  [org.apache.whirr.actions.BootstrapClusterAction] (main)
Bootstrapping cluster
> 2013-02-02 13:55:32,446 INFO  [org.apache.whirr.compute.BootstrapTemplate] (main) Configuring
template for bootstrap-hadoop-datanode_hadoop-tasktracker
> 2013-02-02 13:55:32,494 INFO  [org.apache.whirr.compute.BootstrapTemplate] (main) Configuring
template for bootstrap-hadoop-namenode_hadoop-jobtracker
> 2013-02-02 13:55:32,500 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-1-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node successfully started: nodeId10
> 2013-02-02 13:55:32,500 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-1-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node successfully started: nodeId11
> 2013-02-02 13:55:32,500 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-1-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node successfully started: nodeId12
> 2013-02-02 13:55:32,500 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-1-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node failing to start: nodeId13
> 2013-02-02 13:55:32,502 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-1-thread-4)
[hadoop-namenode, hadoop-jobtracker] - Node failing to start: nodeId20
> 2013-02-02 13:55:32,508 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-1-thread-2)
[hadoop-namenode, hadoop-jobtracker] - Node successfully started: nodeId30
> 2013-02-02 13:55:32,509 INFO  [org.apache.whirr.compute.StartupProcess] (pool-1-thread-3)
Destroying failed nodes [nodeId20]
> 2013-02-02 13:55:32,509 INFO  [org.apache.whirr.compute.StartupProcess] (pool-1-thread-1)
Destroying failed nodes [nodeId13]
> 2013-02-02 13:55:32,510 INFO  [org.apache.whirr.compute.StartupProcess] (pool-1-thread-3)
Destroyed failed nodes []
> 2013-02-02 13:55:32,510 INFO  [org.apache.whirr.compute.StartupProcess] (pool-1-thread-1)
Destroyed failed nodes []
> 2013-02-02 13:55:32,570 WARN  [org.apache.whirr.ClusterSpec] (main) Please use provider
"aws-ec2" instead of "ec2"
> 2013-02-02 13:55:32,574 INFO  [org.apache.whirr.actions.BootstrapClusterAction] (main)
Bootstrapping cluster
> 2013-02-02 13:55:32,575 INFO  [org.apache.whirr.compute.BootstrapTemplate] (main) Configuring
template for bootstrap-hadoop-datanode_hadoop-tasktracker
> 2013-02-02 13:55:32,586 INFO  [org.apache.whirr.compute.BootstrapTemplate] (main) Configuring
template for bootstrap-hadoop-namenode_hadoop-jobtracker
> 2013-02-02 13:55:32,586 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node successfully started: nodeId10
> 2013-02-02 13:55:32,587 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node failing to start: nodeId11
> 2013-02-02 13:55:32,587 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node failing to start: nodeId12
> 2013-02-02 13:55:32,587 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node failing to start: nodeId13
> 2013-02-02 13:55:32,589 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node successfully started: nodeId20
> 2013-02-02 13:55:32,590 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node failing to start: nodeId21
> 2013-02-02 13:55:32,590 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-datanode, hadoop-tasktracker] - Node failing to start: nodeId22
> 2013-02-02 13:55:32,592 INFO  [org.apache.whirr.compute.StartupProcess] (pool-2-thread-1)
Destroying failed nodes [nodeId11, nodeId12, nodeId13, nodeId22, nodeId21]
> 2013-02-02 13:55:32,593 INFO  [org.apache.whirr.compute.StartupProcess] (pool-2-thread-1)
Destroyed failed nodes []
> 2013-02-02 13:55:32,595 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-2-thread-2)
[hadoop-namenode, hadoop-jobtracker] - Node successfully started: nodeId30
> 2013-02-02 13:55:32,686 WARN  [org.apache.whirr.ClusterSpec] (main) Please use provider
"aws-ec2" instead of "ec2"
> 2013-02-02 13:55:32,696 INFO  [org.apache.whirr.actions.BootstrapClusterAction] (main)
Bootstrapping cluster
> 2013-02-02 13:55:32,696 INFO  [org.apache.whirr.compute.BootstrapTemplate] (main) Configuring
template for bootstrap-puppet:module::manifest_something-else
> 2013-02-02 13:55:32,703 INFO  [org.apache.whirr.actions.BootstrapClusterActionTest] (pool-3-thread-2)
[puppet:module::manifest, something-else] - Node successfully started: nodeId10
> 2013-02-02 13:55:32,727 WARN  [org.apache.whirr.ClusterSpec] (main) Please use provider
"aws-ec2" instead of "ec2"
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.391 sec
> Running org.apache.whirr.actions.CleanupClusterActionTest
> 2013-02-02 13:55:33,737 INFO  [org.apache.whirr.actions.BootstrapClusterAction] (main)
Bootstrapping cluster
> 2013-02-02 13:55:33,738 INFO  [org.apache.whirr.compute.BootstrapTemplate] (main) Configuring
template for bootstrap-noop1_noop3
> 2013-02-02 13:55:33,765 ERROR [org.apache.whirr.ClusterController] (main) Unable to start
the cluster. Terminating all nodes.
> java.util.NoSuchElementException: no image matched predicate: And(nullEqualToIsParentOrIsGrandparentOfCurrentLocation(),And(osFamily(ubuntu),osDescription(^(?!.*(daily|testing)).*ubuntu-images.*$),osVersion(10.04)))
> 	at org.jclouds.compute.domain.internal.TemplateBuilderImpl.throwNoSuchElementExceptionAfterLoggingImageIds(TemplateBuilderImpl.java:722)
> 	at org.jclouds.compute.domain.internal.TemplateBuilderImpl.findSupportedImages(TemplateBuilderImpl.java:696)
> 	at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:678)
> 	at org.apache.whirr.compute.BootstrapTemplate.build(BootstrapTemplate.java:77)
> 	at org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:101)
> 	at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:127)
> 	at org.apache.whirr.ClusterController.bootstrapCluster(ClusterController.java:137)
> 	at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:113)
> 	at org.apache.whirr.actions.ScriptBasedClusterActionTest.setUp(ScriptBasedClusterActionTest.java:113)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> 	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> 	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
> 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
> 2013-02-02 13:55:33,765 WARN  [org.apache.whirr.state.ClusterStateStoreFactory] (main)
No cluster state is going to be persisted. There is no easy way to retrieve instance roles
after launch.
> 2013-02-02

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message