hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1820) InputSampler does not create a deep copy of the key object when creating a sample, which causes problems with some formats like SequenceFile<Text,Text>
Date Thu, 08 Jul 2010 00:12:50 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Douglas updated MAPREDUCE-1820:
-------------------------------------

           Status: Resolved  (was: Patch Available)
     Hadoop Flags: [Reviewed]
    Fix Version/s: 0.22.0
       Resolution: Fixed

+1

I committed this. Thanks, Alex!

> InputSampler does not create a deep copy of the key object when creating a sample, which
causes problems with some formats like SequenceFile<Text,Text>
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1820
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1820
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Alex Kozlov
>            Assignee: Alex Kozlov
>             Fix For: 0.22.0
>
>         Attachments: M1820-4.patch, M1820-5.patch, MAPREDUCE-1820-2.patch, MAPREDUCE-1820-3.patch,
MAPREDUCE-1820.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> I tried to use the InputSampler on a SequenceFile<Text,Text> and found that it
comes up with duplicate keys in the sample.  The problem was tracked down to the fact that
the Text object returned from the reader is essentially a wrapper pointing to a byte array,
which changes as the sequence file reader progresses.  There was also a bug in that the reader
should be initialized before the use.  The am attaching a patch that fixes both of the issues.
 --Alex K

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message