hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5882) Progress is not updated when the New Reducer is running reduce phase
Date Fri, 29 May 2009 08:14:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714353#action_12714353
] 

Sharad Agarwal commented on HADOOP-5882:
----------------------------------------

Changing the ReduceContext and having the constructor with null progress object does not seem
very elegant. Would it work, if we just intercept RawKeyValueIterator.next()?  Something like:
{code}
final RawKeyValueIterator rawIter = rIter;
    rIter = new RawKeyValueIterator() {
      public void close() throws IOException {
        rawIter.close();
      }
      public DataInputBuffer getKey() throws IOException {
        return rawIter.getKey();
      }
      public Progress getProgress() {
        return rawIter.getProgress();
      }
      public DataInputBuffer getValue() throws IOException {
        return rawIter.getValue();
      }
      public boolean next() throws IOException {
        reducePhase.set(rawIter.getProgress().getProgress());
        return rawIter.next();
      }
    };
{code}
This looks much simpler to me.


> Progress is not updated when the New Reducer is running reduce phase
> --------------------------------------------------------------------
>
>                 Key: HADOOP-5882
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5882
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jothi Padmanabhan
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.20.1, 0.21.0
>
>         Attachments: patch-5882.txt
>
>
> The old reducer calls informReduceProgress on a key change to update the progress. This
call or equivalent should be called for the new reducer as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message