Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <25120836.1180037896435.JavaMail.jira@brutus>
Date: Thu, 24 May 2007 13:18:16 -0700 (PDT)
From: "Doug Cutting (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-1425) Rework the various programs in
 'examples' to extend ToolBase
In-Reply-To: <24524441.1179957676135.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498806 ] 

Doug Cutting commented on HADOOP-1425:
--------------------------------------

Oops.  You're right.  The conf was handled correctly in your examples all along.  Don't know how I missed that.  Sorry.

>  Do you see any functional difference between above example in DFSAdmin.java and the following code?

The latter has twice as many lines?  Also, you can't reference 'this' in a static method, and the config in that example is never visible to the instance.

An advantage of extending ToolBase is that it causes tools to implement the Tool interface.  If someone were ever to, e.g., implement a Hadoop command shell, then Tool#run() would be what's desired, not a static main method.  This might also simplify things like HADOOP-435, where we replace the command dispatch in bin/hadoop with Java.  Might it also be nice if you could, e.g., specify multiple commands per line, something like: 'bin/hadoop fs -cp foo bar \; job -submit my.job'?  Maybe I'm just grasping at straws now...

Why does ToolBase#doMain() take a Configuration anyway?  If we added a version of 'ToolBase#doMain(String[])' method that creates the Configuration, then tools could look as simple as:
{code}
public class Foo extends ToolBase {
  public run(String[] args) { ... }
  public static void main(String[] argv) throws Exception {
    System.exit(new Foo().doMain(argv));
  }
}
{code}
I can see no way of eliminating the boilerplate 'main' implementation.

If we don't subclass ToolBase, then the simplest I can imagine is:
{code}
public class Foo implements Tool extends Configured {
  public Foo(Configuration) { super(conf); }
  public run(String[] args) { ... }
  public static void main(String[] argv) throws Exception {
    Configuration conf = new Configuration();
    String[] options = ToolBase.parseGeneralOptions(conf, argv);
    System.exit(new Foo(conf).run(options));
  }
}
{code}
Can you make it simpler?  Note that 'implements Tool' and 'extends Configured' are not required.  If 'extends Configured' is dropped however then a 'private Configuration' field would need to be added and set in the constructor.

> Rework the various programs in 'examples' to extend ToolBase 
> -------------------------------------------------------------
>
>                 Key: HADOOP-1425
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1425
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: examples
>            Reporter: Arun C Murthy
>         Assigned To: Enis Soztutar
>            Priority: Minor
>             Fix For: 0.14.0
>
>
> Ensuring all 'examples' extend ToolBase will make it easy to tweak various config params (via -D switches for e.g.) while running the programs... 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.