hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1905) Context.setStatus() and progress() api are ignored
Date Fri, 06 Aug 2010 08:54:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895995#action_12895995
] 

Vinod K V commented on MAPREDUCE-1905:
--------------------------------------

Functionality wise, the patch fixes the current bug and looks fine.

But I sense something wrong. The new TaskAttemptContextImpl doesn't know about reporter even
though the old org.apache.hadoop.mapred.TaskAttemptContextImpl knows about it.
 - progress() calls are made throughout, but they may be getting reported because either the
calls are made on the the old context object (in Task) or directly on the old Reporter class
(in Shuffle etc). So, no bug here as of now, but once we try switching to new API context
objects in the implementation of Task etc, we may encounter new bugs.
 - setStatus() on the new context object are made in several RecordReaders - from the code,
it seems these status updates will not get reported because of the lack of reporter.
 - New Mapper/Reducer classes also make setStatus() calls but on TaskInputOutputContextImpl
which, though knows about the reporter, doesn't override the method from TaskAttemptContextImpl.
This will be fixed by the current patch.

May be, then, the correct and complete solution is to let TaskAttemptContextImpl know about
reporter (via constructor), do setStatus() and progress() in TaskAttemptContextImpl itself
and not override them in TaskInputOutputContextImpl? This will be slightly more changes, so
we can confirm from someone who knows this area well.

> Context.setStatus() and progress() api are ignored
> --------------------------------------------------
>
>                 Key: MAPREDUCE-1905
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1905
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.21.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>         Attachments: patch-1905.txt
>
>
> TaskAttemptContext.setStatus() and progress() were overriden in TaskInputOutputContext,
inbranch 0.20, to call the underlying reporter apis. But the methods are no more over-riden
in TaskInputOutputContextImpl after MAPREDUCE-954.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message