hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Klochkov (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6696) Use contrib/cloud scripts for hardware cluster deployment
Date Fri, 09 Apr 2010 12:05:51 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Klochkov updated HADOOP-6696:
------------------------------------

    Attachment: HADOOP-6696.patch

Attached a patch. 

Some notes on this implementation:

1. It enables Hadoop service deployment only, it doesn't work for zookeeper. 

2. Configuration is done in $HOME/.hadoop-cloud/clusters.cfg. 

Sample config is below. It's set to run jobtracker, namenode and secondary namenode on 172.16.128.203,
and run 2 slaves on 172.16.128.206 and 172.16.128.239.  Parameter NUM_SLAVES of the launch-cluster
command becomes meaningless in case of fixed provider. README is not modified.

[fixed-cluster]
cloud_provider=fixed
image_id=
instance_type=
key_name=
availability_zone=
public_key=/keys/location/hadoop.pem
private_key=/keys/location/hadoop.pem
ssh_options=-i %(private_key)s -o StrictHostKeyChecking=no
nodes.nn_snn_jt=172.16.128.203
nodes.dn_tt=172.16.128.206, 172.16.128.239
env=MAX_MAP_TASKS=2
env=MAX_REDUCE_TASKS=1

3. Files hadoop-ec2-init-remote.sh and hadoop-fixed-init-remote.sh in "data" dir are almost
identical. Obviously it makes sense to extract common code in a separate scripts, and later
use it by other providers too.

> Use contrib/cloud scripts for hardware cluster deployment
> ---------------------------------------------------------
>
>                 Key: HADOOP-6696
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6696
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: contrib/cloud
>            Reporter: Andrew Klochkov
>            Assignee: Andrew Klochkov
>         Attachments: HADOOP-6696.patch
>
>
> contrib/cloud scripts can be used as an easy way to deploy Hadoop on a hardware cluster,
by implementing a "fixed" provider. Here "fixed" means it uses a preconfigured set of hosts
to deploy to. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message