hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4868) Split the hadoop script into 3 parts
Date Tue, 06 Jan 2009 17:26:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12661213#action_12661213

Doug Cutting commented on HADOOP-4868:

> Option 3 looks to be simpler.

I agree.  That's the sort of thing I had in mind.  Nutch does something similar to include
Hadoop in its builds.

> One way could be by some variable CMD_DISPATCHER_CLASS which gets overridden in the mapred
and hdfs.

I was thinking this would be done based on the name of the primary sub-command (fs, job, pipes,
etc.).  We're already proposing to break back-compatibility, changing 'bin/hadoop job' to
'bin/hadoop-mapred job'.  Under my proposal this might instead become 'bin/hadoop mapred job'
(effectively just removing the dash).  Note that 'bin/hadoop fs' could remain unchanged, since
we have a package named 'fs'.

We could preserve 100% compatibilty by placing all command dispatchers under org.apache.hadoop.command.
 So the 'job' command dispatcher could be org.apache.hadoop.command.job.Command or somesuch.

> Just having this won't be sufficient as we need to print help messages listing all the
available commands.

To list all available commands we can scan the classpath (java.class.path), and, for each
file or directory, scan it for org.apache.hadoop.command sub-packages.

> Split the hadoop script into 3 parts
> ------------------------------------
>                 Key: HADOOP-4868
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4868
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>         Attachments: 4868_v1.patch, 4868_v2.patch
> We need to split the bin/hadoop into 3 parts for core, mapred and hdfs. This will enable
us to distribute the individual scripts with each component.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message