[ https://issues.apache.org/jira/browse/WHIRR-212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681521#comment-13681521
]
Jakob Homan commented on WHIRR-212:
-----------------------------------
it's not, but the Kafka jars still aren't in Maven central. That should change with the imminent
0.8 release...
> Add Kafka as a service
> ----------------------
>
> Key: WHIRR-212
> URL: https://issues.apache.org/jira/browse/WHIRR-212
> Project: Whirr
> Issue Type: New Feature
> Components: new service
> Reporter: Jakob Homan
> Assignee: Jakob Homan
>
> Kafka is a a distributed pub/sub system recently open sourced by LinkedIn. The code
repo is located [here|https://github.com/kafka-dev/kafka] and and a good technical overview
can be read [here|http://sna-projects.com/kafka/design.php]. It would be good to be able
to deploy and manage it via Whirr.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681084#comment-13681084
]
Russell Jurney commented on WHIRR-212:
--------------------------------------
So I take it the incubator release is no longer holding this back?
> Add Kafka as a service
> ----------------------
>
> Key: WHIRR-212
> URL: https://issues.apache.org/jira/browse/WHIRR-212
> Project: Whirr
> Issue Type: New Feature
> Components: new service
> Reporter: Jakob Homan
> Assignee: Jakob Homan
>
> Kafka is a a distributed pub/sub system recently open sourced by LinkedIn. The code
repo is located [here|https://github.com/kafka-dev/kafka] and and a good technical overview
can be read [here|http://sna-projects.com/kafka/design.php]. It would be good to be able
to deploy and manage it via Whirr.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Werner Buck created WHIRR-728:
---------------------------------
Summary: Empty locationid for vanilla openstack results in NullPointerException
Key: WHIRR-728
URL: https://issues.apache.org/jira/browse/WHIRR-728
Project: Whirr
Issue Type: Bug
Components: service/zookeeper
Affects Versions: 0.8.2
Environment: Openstack Nova (Vanilla)
Options:
whirr.instance-templates=2 zookeeper
whirr.provider = openstack-nova
whirr.hardware-id = RegionOne/15
whirr.image-id = RegionOne/b647a9cd-99fa-4247-b775-dc0198f3777f (ubuntu lucid cloud image)
jclouds.keystone.tenant-id = a8a405cb510e4607b3f31cf002b60a28
whirr.endpoint = http://ip:5000/v2.0
whirr.identity = demo:admin
whirr.credential = password
jclouds.openstack-nova.auto-create-floating-ips = true
jclouds.openstack-nova.auto-generate-keypairs = true
Reporter: Werner Buck
Priority: Trivial
In the whirr version 0.8.2 a NullPointerException occurs with configureServices with zookeeper
during firewall operations.
Setting location-id to RegionOne fixes it.
Reporting this because I don't know if this is a bug or proper behaviour. I would expect the
location-id to be RegionOne taken from whirr.image-id or whirr.hardware-id if location-id
is not set?
Also all my whirr-runs require jclouds.openstack-nova.auto-generate-keypairs to be set to
true, otherwise an authorization exception is raised. Should I specify another issue for this?
Log:
14:02:08.135 [pool-4-thread-2] INFO org.apache.whirr.compute.NodeStarter - Nodes started:
[{id=RegionOne/da02c9df-2026-44d7-a08a-7d48a6e231b9, providerId=da02c9df-2026-44d7-a08a-7d48a6e231b9,
name=whirrtest-1-f8a, location={scope=HOST, id=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961,
description=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961, parent=RegionOne}, group=whirrtest-1,
imageId=RegionOne/b647a9cd-99fa-4247-b775-dc0198f3777f, os={family=unrecognized, name=ubuntu-lucid-64,
description=ubuntu-lucid-64, is64Bit=true}, status=RUNNING, loginPort=22, hostname=whirrtest-1-f8a,
privateAddresses=[10.11.12.2], publicAddresses=[172.25.95.3], hardware={id=RegionOne/15, providerId=15,
name=custom, location={scope=ZONE, id=RegionOne, description=RegionOne, parent=openstack-nova},
processors=[{cores=1.0, speed=1.0}], ram=256, volumes=[{type=LOCAL, size=10.0, bootDevice=true,
durable=true}], supportsImage=ALWAYS_TRUE}, loginUser=root}, {id=RegionOne/518a5a35-9fea-4df7-aa91-ade52406b6a8,
providerId=518a5a35-9fea-4df7-aa91-ade52406b6a8, name=whirrtest-1-17a, location={scope=HOST,
id=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961, description=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961,
parent=RegionOne}, group=whirrtest-1, imageId=RegionOne/b647a9cd-99fa-4247-b775-dc0198f3777f,
os={family=unrecognized, name=ubuntu-lucid-64, description=ubuntu-lucid-64, is64Bit=true},
status=RUNNING, loginPort=22, hostname=whirrtest-1-17a, privateAddresses=[10.11.12.3], publicAddresses=[172.25.95.2],
hardware={id=RegionOne/15, providerId=15, name=custom, location={scope=ZONE, id=RegionOne,
description=RegionOne, parent=openstack-nova}, processors=[{cores=1.0, speed=1.0}], ram=256,
volumes=[{type=LOCAL, size=10.0, bootDevice=true, durable=true}], supportsImage=ALWAYS_TRUE},
loginUser=root}]
14:02:08.139 [ForkJoinPool-1-worker-2] WARN o.a.w.state.ClusterStateStoreFactory - No cluster
state is going to be persisted. There is no easy way to retrieve instance roles after launch.
14:02:08.140 [ForkJoinPool-1-worker-1] INFO org.rugco.test.WhirrLaunchTest - Bootstrapped
following cluster: Cluster{instances=[Instance{roles=[zookeeper], publicIp=172.25.95.3, privateIp=10.11.12.2,
id=RegionOne/da02c9df-2026-44d7-a08a-7d48a6e231b9, nodeMetadata={id=RegionOne/da02c9df-2026-44d7-a08a-7d48a6e231b9,
providerId=da02c9df-2026-44d7-a08a-7d48a6e231b9, name=whirrtest-1-f8a, location={scope=HOST,
id=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961, description=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961,
parent=RegionOne}, group=whirrtest-1, imageId=RegionOne/b647a9cd-99fa-4247-b775-dc0198f3777f,
os={family=unrecognized, name=ubuntu-lucid-64, description=ubuntu-lucid-64, is64Bit=true},
status=RUNNING, loginPort=22, hostname=whirrtest-1-f8a, privateAddresses=[10.11.12.2], publicAddresses=[172.25.95.3],
hardware={id=RegionOne/15, providerId=15, name=custom, location={scope=ZONE, id=RegionOne,
description=RegionOne, parent=openstack-nova}, processors=[{cores=1.0, speed=1.0}], ram=256,
volumes=[{type=LOCAL, size=10.0, bootDevice=true, durable=true}], supportsImage=ALWAYS_TRUE},
loginUser=root}}, Instance{roles=[zookeeper], publicIp=172.25.95.2, privateIp=10.11.12.3,
id=RegionOne/518a5a35-9fea-4df7-aa91-ade52406b6a8, nodeMetadata={id=RegionOne/518a5a35-9fea-4df7-aa91-ade52406b6a8,
providerId=518a5a35-9fea-4df7-aa91-ade52406b6a8, name=whirrtest-1-17a, location={scope=HOST,
id=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961, description=d3781468178d18f8a789261ff3cb9c3ed0c0444837e6d6bb5a896961,
parent=RegionOne}, group=whirrtest-1, imageId=RegionOne/b647a9cd-99fa-4247-b775-dc0198f3777f,
os={family=unrecognized, name=ubuntu-lucid-64, description=ubuntu-lucid-64, is64Bit=true},
status=RUNNING, loginPort=22, hostname=whirrtest-1-17a, privateAddresses=[10.11.12.3], publicAddresses=[172.25.95.2],
hardware={id=RegionOne/15, providerId=15, name=custom, location={scope=ZONE, id=RegionOne,
description=RegionOne, parent=openstack-nova}, processors=[{cores=1.0, speed=1.0}], ram=256,
volumes=[{type=LOCAL, size=10.0, bootDevice=true, durable=true}], supportsImage=ALWAYS_TRUE},
loginUser=root}}]}
14:02:08.566 [ForkJoinPool-1-worker-1] INFO o.a.whirr.service.FirewallManager - Authorizing
firewall ingress to [RegionOne/da02c9df-2026-44d7-a08a-7d48a6e231b9, RegionOne/518a5a35-9fea-4df7-aa91-ade52406b6a8]
on ports [80, 22, 2181] for [213.126.101.146/32]
14:02:08.566 [ForkJoinPool-1-worker-1] INFO o.a.whirr.service.FirewallManager - Authorizing
firewall ingress to [RegionOne/da02c9df-2026-44d7-a08a-7d48a6e231b9, RegionOne/518a5a35-9fea-4df7-aa91-ade52406b6a8]
on ports [2181] for [213.126.101.146/32]
14:02:08.566 [ForkJoinPool-1-worker-1] INFO o.a.whirr.service.FirewallManager - Authorizing
firewall ingress to [RegionOne/da02c9df-2026-44d7-a08a-7d48a6e231b9, RegionOne/518a5a35-9fea-4df7-aa91-ade52406b6a8]
on ports [80, 22, 2181] for [213.126.101.146/32]
java.lang.RuntimeException: java.lang.NullPointerException: arg[0] in {clazz=interface org.jclouds.openstack.nova.v2_0.extensions.SecurityGroupAsyncApi,
method=public abstract com.google.common.base.Optional org.jclouds.openstack.nova.v2_0.NovaApi.getSecurityGroupExtensionForZone(java.lang.String),
args=[null], returnVal=Sync Proxy for: $Proxy94}
at org.rugco.deployer.deployer.WhirrWrapper$$anonfun$launchInstance$1.applyOrElse(WhirrWrapper.scala:80)
at org.rugco.deployer.deployer.WhirrWrapper$$anonfun$launchInstance$1.applyOrElse(WhirrWrapper.scala:71)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:137)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:136)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:29)
at scala.concurrent.impl.ExecutionContextImpl$$anon$3.exec(ExecutionContextImpl.scala:107)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:262)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1478)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Caused by: java.lang.NullPointerException: arg[0] in {clazz=interface org.jclouds.openstack.nova.v2_0.extensions.SecurityGroupAsyncApi,
method=public abstract com.google.common.base.Optional org.jclouds.openstack.nova.v2_0.NovaApi.getSecurityGroupExtensionForZone(java.lang.String),
args=[null], returnVal=Sync Proxy for: $Proxy94}
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:235)
at org.jclouds.openstack.v2_0.functions.PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.apply(PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.java:70)
at org.jclouds.openstack.v2_0.functions.PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.apply(PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.java:46)
at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:159)
at com.sun.proxy.$Proxy63.getSecurityGroupExtensionForZone(Unknown Source)
at org.apache.whirr.service.FirewallManager.authorizeIngress(FirewallManager.java:310)
at org.apache.whirr.service.FirewallManager.authorizeAllRules(FirewallManager.java:242)
at org.apache.whirr.service.ClusterActionHandlerSupport.handleFirewallRules(ClusterActionHandlerSupport.java:220)
at org.apache.whirr.service.zookeeper.ZooKeeperClusterActionHandler.beforeConfigure(ZooKeeperClusterActionHandler.java:87)
at org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:58)
at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:123)
at org.apache.whirr.ClusterController.configureServices(ClusterController.java:161)
at org.apache.whirr.ClusterController.configureServices(ClusterController.java:153)
at org.rugco.deployer.deployer.WhirrWrapper$$anonfun$configureServices$1.apply(WhirrWrapper.scala:40)
at org.rugco.deployer.deployer.WhirrWrapper$$anonfun$configureServices$1.apply(WhirrWrapper.scala:38)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
... 5 more
14:02:08.576 [ForkJoinPool-1-worker-2] ERROR org.rugco.test.WhirrLaunchTest - *CRITICAL* the
cluster failed to launch and the automated node termination option was not selected, there
might be orphaned nodes.
java.lang.NullPointerException: arg[0] in {clazz=interface org.jclouds.openstack.nova.v2_0.extensions.SecurityGroupAsyncApi,
method=public abstract com.google.common.base.Optional org.jclouds.openstack.nova.v2_0.NovaApi.getSecurityGroupExtensionForZone(java.lang.String),
args=[null], returnVal=Sync Proxy for: $Proxy94}
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:235) ~[guava-13.0.jar:na]
at org.jclouds.openstack.v2_0.functions.PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.apply(PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.java:70)
~[openstack-keystone-1.5.8.jar:1.5.8]
at org.jclouds.openstack.v2_0.functions.PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.apply(PresentWhenExtensionAnnotationNamespaceEqualsAnyNamespaceInExtensionsSet.java:46)
~[openstack-keystone-1.5.8.jar:1.5.8]
at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:159) ~[jclouds-core-1.5.8.jar:1.5.8]
at com.sun.proxy.$Proxy63.getSecurityGroupExtensionForZone(Unknown Source) ~[na:na]
at org.apache.whirr.service.FirewallManager.authorizeIngress(FirewallManager.java:310) ~[whirr-core-0.8.2.jar:0.8.2]
at org.apache.whirr.service.FirewallManager.authorizeAllRules(FirewallManager.java:242) ~[whirr-core-0.8.2.jar:0.8.2]
at org.apache.whirr.service.ClusterActionHandlerSupport.handleFirewallRules(ClusterActionHandlerSupport.java:220)
~[whirr-core-0.8.2.jar:0.8.2]
at org.apache.whirr.service.zookeeper.ZooKeeperClusterActionHandler.beforeConfigure(ZooKeeperClusterActionHandler.java:87)
~[whirr-zookeeper-0.8.2.jar:0.8.2]
at org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:58)
~[whirr-core-0.8.2.jar:0.8.2]
at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:123)
~[whirr-core-0.8.2.jar:0.8.2]
at org.apache.whirr.ClusterController.configureServices(ClusterController.java:161) ~[whirr-core-0.8.2.jar:0.8.2]
at org.apache.whirr.ClusterController.configureServices(ClusterController.java:153) ~[whirr-core-0.8.2.jar:0.8.2]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Ashish Paliwal created WHIRR-727:
------------------------------------
Summary: Add Apache S4 as a Service
Key: WHIRR-727
URL: https://issues.apache.org/jira/browse/WHIRR-727
Project: Whirr
Issue Type: New Feature
Components: new service
Affects Versions: 0.8.2
Reporter: Ashish Paliwal
Add support for Apache S4 as a Service
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13674219#comment-13674219
]
Swathi V commented on WHIRR-415:
--------------------------------
Will the same thing work for cdh4 for specifying different releases?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Assignee: Alex Heneveld
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415.patch, WHIRR-415.patch, WHIRR-415-whirr.env.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is
to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Ok, Thanks a lot! Best regards -Jeremy From: Andrei Savu [mailto:savu.andrei@gmail.com] Sent: Monday, June 03, 2013 6:43 PM To: user@whirr.apache.org Cc: dev@whirr.apache.org Subject: Re: [ASK]Failed in launch-cluster Please don't run Whirr as root on the local machine. -- Andrei Savu / axemblr.com <http://axemblr.com/> On Mon, Jun 3, 2013 at 1:16 PM, lvsongping <lvsongping@cn.fujitsu.com> wrote: Dear whirr users and whirr developer: I'm trying to develop a cloud server recently and I found whirr is used to support to launch a hadoop cluster based on the cloud server(Amazon EC2 or Rackspace), I tried to executed the command line list in the Quick Start Guide(http://whirr.apache.org/docs/0.8.1/quick-start-guide.html), but it is failed when I start to launch the cluster as "/whirr launch-cluster --config hadoop.properties", The error messages are as follows: Cluster-user != root or don not run as root Help: whirr help launch-cluster Here is my the content of my hadoop.properties: whirr.cluster-name=myhadoopcluster whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker whirr.provider=aws-ec2 whirr.private-key-file=root/.ssh/id_rsa whirr.public-key-file=root/.ssh/id_rsa.pub Need I set the whirr.identity and whirr.credential to the hadoop.properties or what other steps I have set into wrong value Best Regards -Jeremy Lv
Please don't run Whirr as root on the local machine. -- Andrei Savu / axemblr.com On Mon, Jun 3, 2013 at 1:16 PM, lvsongping <lvsongping@cn.fujitsu.com>wrote: > Dear whirr users and whirr developer:**** > > ** ** > > I’m trying to develop a cloud server recently and I found whirr is used > to support to launch a hadoop cluster based on the cloud server(Amazon > EC2 or Rackspace), I tried to executed the command line list in the Quick > Start Guide(http://whirr.apache.org/docs/0.8.1/quick-start-guide.html), > but it is failed when I start to launch the cluster as “/whirr > launch-cluster --config hadoop.properties”, The error messages are as > follows:**** > > Cluster-user != root or don not run as root**** > > Help: whirr help launch-cluster**** > > ** ** > > Here is my the content of my hadoop.properties:**** > > whirr.cluster-name=myhadoopcluster **** > > whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 > hadoop-datanode+hadoop-tasktracker **** > > whirr.provider=aws-ec2**** > > whirr.private-key-file=root/.ssh/id_rsa**** > > whirr.public-key-file=root/.ssh/id_rsa.pub**** > > ** ** > > Need I set the whirr.identity and whirr.credential to the hadoop.properties > or what other steps I have set into wrong value**** > > ** ** > > Best Regards**** > > ** ** > > -Jeremy Lv**** >
Dear whirr users and whirr developer: I'm trying to develop a cloud server recently and I found whirr is used to support to launch a hadoop cluster based on the cloud server(Amazon EC2 or Rackspace), I tried to executed the command line list in the Quick Start Guide(http://whirr.apache.org/docs/0.8.1/quick-start-guide.html), but it is failed when I start to launch the cluster as "/whirr launch-cluster --config hadoop.properties", The error messages are as follows: Cluster-user != root or don not run as root Help: whirr help launch-cluster Here is my the content of my hadoop.properties: whirr.cluster-name=myhadoopcluster whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker whirr.provider=aws-ec2 whirr.private-key-file=root/.ssh/id_rsa whirr.public-key-file=root/.ssh/id_rsa.pub Need I set the whirr.identity and whirr.credential to the hadoop.properties or what other steps I have set into wrong value Best Regards -Jeremy Lv
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671691#comment-13671691
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Did you copy the private key to ~/.ssh/id_rsa? If so, then I can completely see how it worked.
=) I'm sneaking a band-aid for JCLOUDS-105 into jclouds 1.6.1, and will be moving Whirr trunk
to use 1.6.1 once it's out. I don't think anything in identity management has changed in whirr
between 0.8.2 and trunk before my branch, so I'm a little confused as to why it'd work (sorta)
on 0.8.2 but not on my branch/trunk, but I'll poke at it some more later today.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671309#comment-13671309
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
I tested 0.8.2 again (fresh build), same hadoop.properties file and it does bootstrap...so
the ssh key based connection works there. There must be some difference with the 725 branch...
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671154#comment-13671154
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
Your 725 branch never worked for me, but the whirr source did bootstrap....
Great news though, I will try to have a look at it after breakfast (europe time).
I do remember that I had to grab the public key from another instance, because you don't get
it from exoscale. Maybe that's causing the issue...
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670935#comment-13670935
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Ah-ha! I'm still not sure how you ever got it to bootstrap and configure, but at least in
my case, I have a fix. =) JCLOUDS-105 - a really simple flip of logic that, along with specifying
whirr.cloudstack-keypair=foo (which is in a change I'm about to push to the whirr-725 branch),
makes it all work. woo! Of course, the downside is that this means it needs a new version
of jclouds. =)
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670859#comment-13670859
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
So whenever I create an instance without explicitly telling it to use a keypair, it doesn't
get a keypair and Whirr tries to authenticate with the password, which understandably fails.
I just can't get it to stop trying to do password authentication, which is...frustrating.
I'll keep poking at it, and if I keep being stuck, I'll contact exoscale. In the meantime
- can you possibly email your whirr.log? It's got some private info in it (API key/secret
key most notably), so you may want to scrub those. I just can't figure out how it's not trying
password authentication for you...
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670808#comment-13670808
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Aaaaaah - I was trying with an imported keypair. Lemme try again with a generated keypair.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670804#comment-13670804
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
just to avoid confusion. you need to create a keypair manually or via coudstack API, they
send you the private key by email. exoscale automatically places the public key on the instance.
If you try to ssh using the default password (available in the job status result), it won't
work.
I have not tried their cloudinit setup yet.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670764#comment-13670764
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
with whirr-0.8.2 I had just setup:
whirr.private-key-file=${sys:user.home}/.ssh/id_rsa_foobar
whirr.public-key-file=${whirr.private-key-file}.pub
and it was boostrapping fine.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670760#comment-13670760
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Yeah, I was trying the password from startup on the console too. Ok, I guess I need to figure
out how to make jclouds play nicely with CloudStack named keypairs then. =) And they need
to fix whatever's going on there with cloud-init and keys. I'll see if I can find some hack
way to get Whirr+jclouds to use named keypairs - jclouds is expecting to find the private
key for the named keypair somewhere in memory, but it's a rather old credential store that
isn't in use almost anywhere else. I'll see what I can figure out.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sebastien goasguen updated WHIRR-725:
-------------------------------------
Attachment: console.jpg
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670752#comment-13670752
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
ok so I am with you, I see the jclouds-<cluster> sg.
I can create instances fine via the UI and ssh with keys. ssh with password is disabled, you
can only use the password that's returned at job startup (you see it in the job status) for
login via the console.
I attach the console I get, there is an issue with root.ec2 and keys..<surprise>
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: console.jpg, whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670739#comment-13670739
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Ah, yeah, whirr.client-cidrs=0.0.0.0/0 (or whatever value you want there).
And any instance I create, either through Whirr or through the exoscale UI, isn't setting
the password for ubuntu (or root) that it claims to have used, and in fact, when I set one
instance up with a keypair and SSH'd in, I could see that the password hadn't been updated
at all. So something's definitely awry there - and I have yet to figure out a way to use named
keypairs for CloudStack with whirr, so, hey, fun. =)
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670738#comment-13670738
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
my bad, wrong branch. Now I get:
whirr launch-cluster --config hadoop.properties
Running on provider cloudstack using identity mnH5EbKcKeJdJcsSnav7SKoqwNtZFGXOiITNCev5AKAd3ArKkMF5FlpurvguEIwQG_Fn-N0lcRt8AZW4kYV55Q
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException:
you must specify at least one cidr range
at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:128)
at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:69)
at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:59)
at org.apache.whirr.cli.Main.run(Main.java:69)
at org.apache.whirr.cli.Main.main(Main.java:102)
Caused by: java.lang.IllegalArgumentException: you must specify at least one cidr range
I guess there is an option to set in the properties file
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670732#comment-13670732
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Yeah - https://github.com/abayer/whirr/commits/whirr-725, the latest commit is where I am
currently. Last I checked, I was still seeing authentication errors...I'll try again.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670727#comment-13670727
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
are you sure you pushed ? still nothing
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670725#comment-13670725
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
just attached properties file
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sebastien goasguen updated WHIRR-725:
-------------------------------------
Attachment: whirr.properties
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
> Attachments: whirr.properties
>
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670723#comment-13670723
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Whoops, messed up my push - fixed now. Did you set "whirr.use-cloudstack-security-group=true"
in the properties?
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670722#comment-13670722
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
Looks like they had a maintenance window. I tried to pull your latest but there was nothing.
It's still failing boostrap for me, and creating instances in the default sg.
I will attach my properties file.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670534#comment-13670534
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Ok, the branch has been updated again - I was able to get security groups to create/assign/open
port 22, but since I can't get login to work for some reason (I think because of misconfiguration
on their side?), I can't actually get to the firewall manager stage. I think your bootstrap
failures are probably related, based on what I saw in whirr.log. I'll give it another shot
this weekend.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670526#comment-13670526
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Ok, it's something weird on their end not working right. I'll check again later.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670492#comment-13670492
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Hrm - I can't seem to get Whirr to login for the bootstrapping - can you post your whirr config?
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670412#comment-13670412
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Well, bugger. =) I'll see what I can do.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670330#comment-13670330
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
git cloned you repo, built with mvn install, launch the cluster with the new config.
Instances got started in the default group, no jclouds-(cluster-name) got created.
The bootstrap ended failing and the instances got destroyed.
there was no stdout on the whirr launch-cluster command
Unfortunately my java is non-existent
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13670306#comment-13670306
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
Ok let me check that. I sent you some creds for a public cloud you could use...in your gmail
address.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13668901#comment-13668901
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Ok, figured out a bit more - we don't actually want to muck around with the default security
group, and we also aren't already creating a security group. Now, I think, it should - check
the latest I've pushed to https://github.com/abayer/whirr/tree/whirr-725. If you add "whirr.use-cloudstack-security-group=true"
to your config, it *should* create a "jclouds-(cluster name)" security group, if one doesn't
already exist, and add each of your instances to that group. Then, when it gets to dealing
with the ports, it should add all of the appropriate ingress rules to that security group.
Be warned that there's nothing for cleaning up the security group, so it's not at all unlikely
that you could end up with a plethora of security groups floating around. I don't have a good
environment to test this out myself right now, so it's kind of in your hands. =) Let me know
how it goes!
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13668755#comment-13668755
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
haha, of course it's the one I assumed it wasn't. =) I'll try to get basic networking security
groups support in over the next couple days. For reference, what's the name of the security
group that gets created/applied to your instances when you spin them up through Whirr? Or
is just using the default one?
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667937#comment-13667937
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
by the way it's a basic zone setup in such a way that all instances get a public address.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667918#comment-13667918
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
I sent you a promo code to your gmail address.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667914#comment-13667914
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Ok, I've got https://github.com/abayer/whirr/tree/whirr-725 up - added support for firewall
rules for advanced networking in CloudStack. Working on adding basic networking support (e.g.,
security groups) now, but if you're using advanced networking, this should be sufficient.
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667911#comment-13667911
]
sebastien goasguen commented on WHIRR-725:
------------------------------------------
Yes it's public: http://exoscale.ch, let me get you a promo code, I will sent you a DM on
twitter.
-sebastien
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[ https://issues.apache.org/jira/browse/WHIRR-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667899#comment-13667899
]
Andrew Bayer commented on WHIRR-725:
------------------------------------
Don't suppose by any chance this CS 4.0.2 cloud is publicly accessible? =) I only have a CS
3.0.3 cloud with basic networking available for testing, which makes testing this a bit harder
than I'd like. If not, that's cool, I'll bug [~ke4qqq]. =)
> Security Group rules don't get created on CloudStack 4.0.2 provider
> -------------------------------------------------------------------
>
> Key: WHIRR-725
> URL: https://issues.apache.org/jira/browse/WHIRR-725
> Project: Whirr
> Issue Type: Bug
> Components: new provider, service/hadoop
> Affects Versions: 0.8.2
> Environment: Running Whirr 0.8.2 on OSX 10.8.2, compiled from source. Using a
cloudstack 4.0.2 cloud.
> Reporter: sebastien goasguen
>
> Trying to deploy a cdh cluster, the instances start, nodes get configured. But the security
groups rules don't get created/added to the SG. No errors in the whirr.log file.
> Adding the rules by hand, out of band from whirr fixes the issue. Instances get a working
hadoop (at least hadoop fs -ls / works).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira