hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leif Wickland <leifwickl...@gmail.com>
Subject Re: HFiles and MapReduce
Date Mon, 01 Aug 2011 21:15:52 GMT
Thanks for the reply, J-D.

> > 1. Is there any case where it's a bad idea to use HFileOutputFormat
> instead
> > of TableOutputFormat when writing to HBase from MapReduce?
> Can you think of any?

Well, no, but I'm new to this stuff.

> That process isn't atomic, so to be sure you could end up with a
> region failing for some reason (network issues, bug, whatever) and my
> understanding of the code is that it would fail and return immediately
> after any IOException.

What would recovery from that scenario look like?  Would the un-adopted
HFiles remain in the directory that they were written to by the

> You'd need to write an HFileInputFormat, that's pretty much it.

It looks like Tatsuya Kawano offered to write an
in January. Does anyone know if he ended up sharing that?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message