hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerry Lam <chiling...@gmail.com>
Subject Streaming value of (200MB) from a SequenceFile
Date Sat, 30 Mar 2013 17:52:33 GMT
Hi everyone,

I'm having a problem to stream individual key-value pair of 200MB to 1GB
from a MapFile.
I need to stream the large value to an outputstream instead of reading the
entire value before processing because it potentially uses too much memory.

I read the API for MapFile, the next(WritableComparable key, Writable val)
does not return an input stream.

How can I accomplish this?



View raw message