hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant Sharma (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3605) Allow mr commands to be run via bin/hadoop
Date Thu, 29 Dec 2011 07:04:30 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177033#comment-13177033
] 

Prashant Sharma commented on MAPREDUCE-3605:
--------------------------------------------

Arun,

{quote}
Did you test your patch?
{quote}
Yes, But not thoroughly.

{quote}
Prashanth - this needs more work. You need to add YARN & MR jars to classpath etc.
{quote}

 Including YARN & MR jars on classpath wont be necessary as the hadoop script is delegating
the command and its params to mapred shell command. This is similar to executing hdfs command
and since that works out of the box with-out explicitly setting classpath, so a similar behavior
can be expected from this as well.


Besides above. There should exist a "hadoopctl" (A typical script found with most standard
opensource utils) script which provides a central way of executing scripts or doing all the
tasks. Accompanied by a proper man page it would serve as a standard way of starting various
daemons with complete configuration details. Standard because most of the other opensource
tools and even bigger applications have such support. 

A worth-noting advantage it provides is that - with changing of design of internal classes
e.g. deprecation and changes in classpath etc... The hadoopctl script will serve as a complete
abstraction and wont bother end user. Second advantage will be limiting the number of variables
needed to export and Third will be users will have to consult just one script to build there
way of automating hadoop installation by building scripts on the top of hadoopctl by consulting
a simple man page. Segregation of three projects is great but consulting documentations and
script of three different projects to do one thing doesn't that cool. 

Thoughts? 

                
> Allow mr commands to be run via bin/hadoop
> ------------------------------------------
>
>                 Key: MAPREDUCE-3605
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3605
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1
>            Reporter: Ramya Sunil
>             Fix For: 0.23.1
>
>         Attachments: MAPREDUCE-3605.patch
>
>
> MR command line options are not supported in bin/hadoop.
> {noformat}
> bin/hadoop job
> Exception in thread "main" java.lang.NoClassDefFoundError: job
> Caused by: java.lang.ClassNotFoundException: job
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class: job.  Program will exit.
> {noformat}
> A deprecated message like "DEPRECATED: Use of this script to execute mapred command is
deprecated. Instead use the mapred command for it." should be displayed along with the correct
output.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message