hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Kozlov (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-1820) InputSampler does not create a deep copy of the key class when creating a sample, which causes problems with some formats like SequenceFile<Text,Text>
Date Wed, 26 May 2010 22:31:36 GMT
InputSampler does not create a deep copy of the key class when creating a sample, which causes
problems with some formats like SequenceFile<Text,Text>
------------------------------------------------------------------------------------------------------------------------------------------------------

                 Key: MAPREDUCE-1820
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1820
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Alex Kozlov


I tried to use the InputSampler on a SequenceFile<Text,Text> and found that it comes
up with duplicate keys in the sample.  The problem was tracked down to the fact that the Text
object returned from the reader is essentially a wrapper pointing to a byte array, which changes
as the sequence file reader progresses.  There was also a bug in that the reader should be
initialized before the use.  The am attaching a patch that fixes both of the issues.  --Alex
K

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message