hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leon Mergen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3932) Two small improvements to pipes
Date Fri, 15 Aug 2008 17:41:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622937#action_12622937

Leon Mergen commented on HADOOP-3932:


I have created two separate patches for this using svn diff.

Be aware that the second patch, "pipes_interface.patch" actually changes the interface of
the Submitter.submitJob () method, and can very well cause applications to break. However,
I am of the opinion that the name "submitJob ()" for a function that actually does a "runJob
()" on the JobClient was wrong to begin with.

The pipes Submitter class should conform to the mapreduce JobClient's interface naming: runJob
() submits the job to the map/reduce framework and then polls for progress until the job is
complete, and submitJob () should just submit the job to the map/reduce framework in the backrground.

> Two small improvements to pipes
> -------------------------------
>                 Key: HADOOP-3932
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3932
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: pipes
>    Affects Versions: 0.17.1
>         Environment: n/a
>            Reporter: Leon Mergen
>            Assignee: Leon Mergen
>         Attachments: nullpointer_fix.patch, pipes_interface.patch
> Working with the ..mapred.pipes class a bit today, I found one bug and one possible interface
> - Application.java in its constructor assumes that DistributedCache.getLocalCacheFiles
() always returns non-zero, and returns an array with at least one element -- appropriate
checks and exceptions should be thrown here.
> - Submitter.java provides a "submitJob ()" method, yet it acts like it's a runJob method
(and also invokes mapred.JobClient.runClient ()) -- it should provide two interface methods,
one runJob () and one submitJob (), who act just like the JobClient counterparts.
> Here is the small patch that implements both changes, based on the 0.17.1 release source
-- just in case anyone cares for this minor improvement.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message