hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kunal Gupta <ku...@techlead-india.com>
Subject Re: issue using LineRecordReader's nextKeyValue() method
Date Wed, 09 Dec 2009 05:11:31 GMT
Thanks Aaron,
        
        Your last point was absolutely correct and i was facing problem
        because
        of my lack of understanding of this point. I had later figured
        out the
        issue and allocated new memory for each key and value and got
        the issue
        resolved. Thanks for the response..
        
        Aaron i was wondering whether i shall post such 'thanks'
        messages on the
        mapreduce-user mailing list or to the individual person? what do
        you
        suggest?
        
        -Kunal
        

On Mon, 2009-12-07 at 15:51 -0800, Aaron Kimball wrote:
> getCurrentKey() and getCurrentValue() do just that: they return the
> current (k, v) pair i.e. the current line. The notion of the current
> line is not changed by calling these methods.
> 
> nextKeyValue()'s contract is to advance the iterator and return true
> if more data is now available.
> 
> Note that even if the data held in the current key and value objects
> is updated by nextKeyValue(), the LineRR will not create new objects
> to hold those key and value pairs -- it recycles objects for
> efficiency.
> 
> - Aaron
> 
> 
> On Wed, Dec 2, 2009 at 5:45 AM, Kunal Gupta <kunal@techlead-india.com>
> wrote:
>         I am writing my custom InputFormat to read N number of lines
>         per map task.
>         
>         For this I have extended the FileInputFormat and RecordReader
>         classes.
>         
>         In my RecordReader I am using LineRecordReader object to get
>         key value pairs.
>         
>         If I am calling LineRecordReaderObj.nextKeyValue() method
>         multiple times while 
>         
>         handling the same file SPLIT, it goes to the next keyValue but
>         the following functions 
>         
>         return same keyValue every time:
>         
>         getCurrentKey() / getCurrentValue()
>         
>         What may be the possible reason for this behavior?
>         
>          
>         
>         
> 


Mime
View raw message