hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3356) SequenceFile.MergeQueue.merge inadvertently creates merge-outputs in the wrong FileSystem, at times in the InMemoryFileSystem
Date Wed, 07 May 2008 05:45:55 GMT
SequenceFile.MergeQueue.merge inadvertently creates merge-outputs in the wrong FileSystem,
at times in the InMemoryFileSystem
-----------------------------------------------------------------------------------------------------------------------------

                 Key: HADOOP-3356
                 URL: https://issues.apache.org/jira/browse/HADOOP-3356
             Project: Hadoop Core
          Issue Type: Bug
          Components: io
    Affects Versions: 0.16.3
            Reporter: Arun C Murthy
            Assignee: Arun C Murthy
             Fix For: 0.18.0


The offending code is:

{code:title=SequenceFile.java}
            Path outputFile =  lDirAlloc.getLocalPathForWrite(
                                                tmpFilename.toString(),
                                                approxOutputSize, conf);
            LOG.debug("writing intermediate results to " + outputFile);
            Writer writer = cloneFileAttributes(
                                                fs.makeQualified(segmentsToMerge.get(0).segmentPathName),

                                                fs.makeQualified(outputFile), null);
{code}

*fs* is InMemoryFileSystem when ReduceTask.ReduceCopier constructs it... so the wrong FileSystem
is used during intermediate merges.
 


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message