hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-59) support generic command-line options
Date Fri, 23 Jun 2006 18:21:30 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-59?page=all ]

Hairong Kuang updated HADOOP-59:

    Attachment: genericCommand.patch

This patch provides a general framework to support general command options. Tool provides
an interface and ToolBase processes general command options.

General command options supported are
-conf <configuration file>      specify an application configuration file
-D <property=value>               use value for given property
-fs <local|namenode:port>    specify a name node
-jt <local|jobtracker:port>       specify a job tracker

The syntax to run bin/hadoop becomes
bin/hadoop command [general options] [command options and arguments]

The patch also includes changes to four tools, DFSShell, DFSck, JobClient, and CopyFiles to
inherit from ToolBase. 

Examples using general options are
bin/hadoop dfs -fs darwin:8020 -ls /data
bin/hadoop dfs -Dfs.default.name=darwin:8020 -ls /data
bin/hadoop dfs -conf hadoop-site.xml -ls /data
bin/hadoop job -jt darwin:50020 -status job_0001
bin/hadoop distcp -Dfs.default.name=darwin:8020 -Dmapred.job.tracker=darwin:50020 srcurl dsturl
bin/hadoop fsck -fs dawin:8020 /data

Last, the patch includes a junit test for general options.

> support generic command-line options
> ------------------------------------
>          Key: HADOOP-59
>          URL: http://issues.apache.org/jira/browse/HADOOP-59
>      Project: Hadoop
>         Type: Improvement

>   Components: conf
>     Versions: 0.2.0
>     Reporter: Doug Cutting
>     Assignee: Hairong Kuang
>     Priority: Minor
>      Fix For: 0.4.0
>  Attachments: Tool.java, ToolBase.java, bashfile.patch, genericCommand.patch, toolbase.patch
> Hadoop commands should all support some common options.  For example, it should be possible
to specify the namenode, datanode, and, for that matter, any config option, in a generic way.
> This could be implemented with code like:
> public interface Tool extends Configurable {
>   void run(String[] args) throws Exception;
> }
> public class ToolBase implements Tool extends Configured {
>   public final void main(String[] args) throws Exception {
>     Configuration conf = new Configuration();
>     ... parse config options from args into conf ...
>     this.configure(conf);
>     this.run();
>   }
> }
> public MyTool extends ExcecutableBase {
>   public static void main(String[] args) throws Exception {
>     new MyTool().main(args);
>   }
> }
> The general command line syntax could be:
> bin/hadoop [generalOptions] command [commandOptions]
> Where generalOptions are things that ToolBase handles, and only the commandOptions are
passed to Tool.run().  The most important generalOption would be '-D', which would define
name/value pairs that are set in the configuration.  This alone would permit folks to set
the namenode, datanode, etc.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message