Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <977740008.1218618885033.JavaMail.jira@brutus>
Date: Wed, 13 Aug 2008 02:14:45 -0700 (PDT)
From: "Arun C Murthy (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-2168) Pipes with a C++ record reader does
 not update progress in the map until it is 100%
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622142#action_12622142 ] 

Arun C Murthy commented on HADOOP-2168:
---------------------------------------

This is turning out to be harder than I expected - the problem being that only MapTask.TrackerRecordReader updates progress and in this case the Pipes application isn't looping through the Java RecordReader...

Possible solutions:
1. Add a 'setProgress' api to org.apache.hadoop.mapred.Reporter - this is quite dangerous since applications can potentially use this to throw the Scheduler off.
2. Use a dummy InputFormat/RecordReader along with a specialized Reporter which can communicate the progress to the RecordReader and hence the rest of the framework.

Given the dangers of the first approach, Owen/me agree we should go with the second for now. Thoughts?

> Pipes with a C++ record reader does not update progress in the map until it is 100%
> -----------------------------------------------------------------------------------
>
>                 Key: HADOOP-2168
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2168
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: pipes
>            Reporter: Owen O'Malley
>            Assignee: Arun C Murthy
>
> Currently C++ maps with C++ record readers do not update the progress, even if the application has updated it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.