hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: One map task to two HFiles
Date Wed, 25 May 2011 04:04:01 GMT
On Tue, May 24, 2011 at 2:26 PM, Jon Stewart
<jon@lightboxtechnologies.com> wrote:
> I have enough control over the keys for the primary table that the map
> task could write rows to the primary table in order, making it
> map-side only (assuming one HFile per task). The map task could then
> emit KeyValue objects for the secondary hash table and let
> HFileOutpuFormat/KeyValueSortReducer do its thing.
> The question is, how do I write an HFile from a map task?
> HFile.Writer? What are the gotchas?

Nothing fancier than open on map task init, append, append, append,
inside in the map, and then be sure to close it in the map close.
Study HFileOutputFormat I'd say.


View raw message