hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Explosion in datasize using HBase as a MR sink
Date Tue, 04 Jun 2013 23:07:29 GMT
On Tue, Jun 4, 2013 at 9:58 PM, Rob Verkuylen <rob@verkuylen.net> wrote:

> Finally fixed this, my code was at fault.
>
> Protobufs require a builder object which was a (non static) protected
> object in an abstract class all parsers extend. The mapper calls a parser
> factory depending on the input record. Because we designed the parser
> instances as singletons, the builder object in the abstract class got
> reused and all data got appended to the same builder. Doh! This only shows
> up in a job, not in single tests. Ah well, I've learned a lot  :)
>
>
Thanks for updating the list Rob.

Yours is a classic except it is first time I've heard of someone
protobufing it..  Usually it is a reuse of an Hadoop Writable instance
accumulating....

St.Ack

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message