hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell" <apurt...@apache.org>
Subject Re: [jira] Commented: (HBASE-1961) HBase EC2 scripts
Date Sat, 14 Nov 2009 01:06:54 GMT
Stack,

Can you please add a comment on the issue on my behalf? Can't seem to do it from the blackberry.
The scripts work. I don't want the amz guys going on the issue thinking otherwise.

Thanks,

    - Andy

-----Original Message-----
From: "Andrew Purtell" <apurtell@apache.org>
Date: Sat, 14 Nov 2009 01:00:54 
To: Michael Stack<stack@duboce.net>; <hbase-dev@hadoop.apache.org>
Subject: Re: [jira] Commented: (HBASE-1961) HBase EC2 scripts

You need both the api and ami tools. Thought you had one but not the other. You need both.
Click through both links in the readme and the comment on the issue. Also see the comment
on the issue about the help. The help is correct. It is for that script. See the if-else switch
inside. It is not help for the *-hbase-* worker scripts.

-----Original Message-----
From: "stack (JIRA)" <jira@apache.org>
Date: Sat, 14 Nov 2009 00:45:39 
To: <apurtell@apache.org>
Subject: [jira] Commented: (HBASE-1961) HBase EC2 scripts


    [ https://issues.apache.org/jira/browse/HBASE-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777775#action_12777775
] 

stack commented on HBASE-1961:
------------------------------

This latest tarball still has usage that does not mention hbase:

{costack@aron:~/checkouts/workspace/trunk/src/contrib/ec2$ ./bin/hbase-ec2 
Usage: hbase-ec2 COMMAND
where COMMAND is one of:
  list                                  list all running HBase EC2 clusters
  launch-cluster <name> <slaves> <zoos> launch a HBase cluster
  launch-zookeeper <name> <zoos>        launch the zookeeper quorum
  launch-master  <name>                 launch or find a cluster master
  launch-slaves  <name> <slaves>        launch the cluster slaves
  terminate-cluster  <name>             terminate all HBase EC2 instances
  delete-cluster <name>                 clean up after a terminated cluster
  login  <name|instance id>             login to the master node
  screen <name|instance id>             start or attach 'screen' on the master
  proxy  <name|instance id>             start a socks proxy on localhost:6666
  push   <name> <file>                  scp a file to the master node
  <shell cmd> <group|instance id>       execute a command on the master
  create-image                          create a HBase AMIde}

{code}

.. it should be 'launch-hbase-cluster' instead of 'launch-cluster'? etc.

I did the apt-get install of ami tools.  Got this error when I tried below command:

{code}

{code$ ./bin/launch-hbase-cluster x 3 3
Creating/checking security groups
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
34: ec2-describe-group: command not found
Creating group x-master
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
37: ec2-add-group: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
38: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
39: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
51: ec2-describe-group: command not found
Creating group x
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
54: ec2-add-group: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
55: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
56: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
65: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
66: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
73: ec2-describe-group: command not found
Creating group x-zookeeper
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
76: ec2-add-group: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
77: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
78: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
80: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
81: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
82: ec2-authorize: command not found
/home/stack/checkouts/workspace/trunk/src/contrib/ec2/bin/init-hbase-cluster-secgroups: line
83: ec2-authorize: command not found
}

> HBase EC2 scripts
> -----------------
>
>                 Key: HBASE-1961
>                 URL: https://issues.apache.org/jira/browse/HBASE-1961
>             Project: Hadoop HBase
>          Issue Type: New Feature
>         Environment: Amazon AWS EC2
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>             Fix For: 0.21.0, 0.20.3
>
>         Attachments: ec2-contrib.tar.gz
>
>
> Attached tarball is a clone of the Hadoop EC2 scripts, modified significantly to start
up a HBase storage only cluster on top of HDFS backed by instance storage. 
> Tested with the HBase 0.20 branch but should work with trunk also. Only the AMI create
and launch scripts are tested. Will bring up a functioning HBase cluster. 
> Do "create-hbase-image c1.xlarge" to create an x86_64 AMI, or "create-hbase-image c1.medium"
to create an i386 AMI.  Public Hadoop/HBase 0.20.1 AMIs are available:
>     i386: ami-c644a7af
>     x86_64: ami-f244a79b
> launch-hbase-cluster brings up the cluster: First, a small dedicated ZK quorum, specifiable
in size, default of 3. Then, the DFS namenode (formatting on first boot) and one datanode
and the HBase master. Then, a specifiable number of slaves, instances running DFS datanodes
and HBase region servers.  For example:
> {noformat}
>     launch-hbase-cluster testcluster 100 5
> {noformat}
> would bring up a cluster with 100 slaves supported by a 5 node ZK ensemble.
> We must colocate a datanode with the namenode because currently the master won't tolerate
a brand new DFS with only namenode and no datanodes up yet. See HBASE-1960. By default the
launch scripts provision ZooKeeper as c1.medium and the HBase master and region servers as
c1.xlarge. The result is a HBase cluster supported by a ZooKeeper ensemble. ZK ensembles are
not dynamic, but HBase clusters can be grown by simply starting up more slaves, just like
Hadoop. 
> hbase-ec2-init-remote.sh can be trivially edited to bring up a jobtracker on the master
node and task trackers on the slaves.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Mime
View raw message