hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6392) Run namenode and jobtracker on separate EC2 instances
Date Wed, 09 Dec 2009 00:33:18 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tom White updated HADOOP-6392:
------------------------------

    Attachment: HADOOP-6392.patch

Here's a patch that removes the strong requirement to have "master" and "slave" types. Instead,
instances have one or more roles, which are stored in EC2 as security groups. The instance
uses knowledge of its role to start the relevant daemons at boot time - for example the "nn"
role will start up the namenode.

Here's how you launch one master node (running a namenode, secondary namenode, and a jobtracker)
and 10 datanodes/tasktrackers:

{code}
hadoop-ec2 launch-cluster my-hadoop-cluster 1 nn,snn,jt 10 dn,tt
{code}

The following command will launch a namenode and a separate jobtracker (and 10 datanodes/tasktrackers).

{code}
hadoop-ec2 launch-cluster my-hadoop-cluster 1 nn,snn 1 jt 10 dn,tt
{code}

This patch is fairly large, but there is little new code, since it is mainly refactoring.
Previously, the logic for each command was spread between cli.py and commands.py. Now, cli.py
is responsible only for command line parsing, and it delegates to service.py, which contains
the class HadoopService.py for the Hadoop-specific logic.

> Run namenode and jobtracker on separate EC2 instances
> -----------------------------------------------------
>
>                 Key: HADOOP-6392
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6392
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: contrib/ec2
>            Reporter: Tom White
>            Assignee: Tom White
>         Attachments: HADOOP-6392.patch
>
>
> Replace concept of "master" with that of "namenode" and "jobtracker". Still need to be
able to run both on one node, of course.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message