hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Armstrong <john.armstr...@ccri.com>
Subject Re: SequenceFile.Reader
Date Thu, 02 Jun 2011 22:49:20 GMT
On Thu, 2 Jun 2011 15:43:37 -0700, Mark question <markq2011@gmail.com>
>  Does anyone knows if :  SequenceFile.next(key) is actually not reading
> value into memory????

I think what you're confused by is something I stumbled upon quite by
accident.  The secret is that there is actually only ONE Key object that
the RecordReader presents to you.  The next() method doesn't create a new
Key object (containing the new data) but actually just loads the new data
into the existing Key object.

The only place I've seen that you absolutely must remember these unusual
semantics is when you're trying to copy keys or values for some reason, or
to iterate over the Iterable of values more than once.  In these cases you
must make defensive copies because otherwise you'll just git a big list of
copies of the same Key, containing the last Key data you saw.


View raw message