Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <23573883.1164913943399.JavaMail.jira@brutus>
Date: Thu, 30 Nov 2006 11:12:23 -0800 (PST)
From: "Sanjay Dahiya (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Updated: (HADOOP-728) Map-reduce task does not produce
 correct results when -reducer NONE is specified through streaming
In-Reply-To: <30411461.1163635597099.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

     [ http://issues.apache.org/jira/browse/HADOOP-728?page=all ]

Sanjay Dahiya updated HADOOP-728:
---------------------------------

    Attachment: Hadoop-728.patch

This patch fixes a few other issues as well - 
1. Some map tasks writing to DFS were generating incomplete files, added flush() in MROutputThread
2. Added extra checks in TaskTracker and PhasedFileSystem to check for temporary file existence before attempting delete. This avoids extra lines in namenode logs. 
3. Changed a variable name, sideEffectURI_ is now finalOutputURI and is used to get correct file name. 

off topic : I was not aware that there are real languages written bottom to top http://en.wikipedia.org/wiki/Writing_system#Directionality 

> Map-reduce task does not produce correct results when -reducer NONE is specified through streaming
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-728
>                 URL: http://issues.apache.org/jira/browse/HADOOP-728
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/streaming
>            Reporter: dhruba borthakur
>         Assigned To: Sanjay Dahiya
>         Attachments: Hadoop-728.patch, Hadoop-728.patch
>
>
> a) a file is create for the output instead of a directory.
> b) there is no way to understand what is going on from the client output
> I can produce an example for you, if you like -- but the behavior is consistent, so $HSTREAM -mapper /bin/cat -reducer NONE should show the problem
> ~

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira