hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Kang <weliam.cl...@gmail.com>
Subject Re: Limits on HBase
Date Tue, 07 Sep 2010 05:40:01 GMT
Hi JG,
Thanks for your reply. As far as I have read in Hbase's documentation and
wiki, the cell size is not supposed to be larger than 10 MB. For the row, I
am not quite sure, but it looks like 256 MB is the upper limit. I am
considering store some binary data used to be stored in RDBM blob field. The
size of those binary objects may vary from hundreds of KB to hundreds of MB.
What would be a good way to use Hbase for it? We really want to use hbase to
avoid that scaling problem.
Many thanks.


William

On Mon, Sep 6, 2010 at 7:10 PM, Jonathan Gray <jgray@facebook.com> wrote:

> I'm not sure what you mean by "optimized cell size" or whether you're just
> asking about practical limits?
>
> HBase is generally used with cells in the range of tens of bytes to
> hundreds of kilobytes.  However, I have used it with cells that are several
> megabytes, up to about 50MB.  Up at that level, I have seen some weird
> performance issues.
>
> The most important thing is to be sure to tweak all of your settings.  If
> you have 20MB cells, you need to be sure to increase the flush size beyond
> 64MB and the split size beyond 256MB.  You also need enough memory to
> support all this large object allocation.
>
> And of course, test test test.  That's the easiest way to see if what you
> want to do will work :)
>
> When you run into problems, e-mail the list.
>
> As far as row size is concerned, the only issue is that a row can never
> span multiple regions so a given row can only be in one region and thus be
> hosted on one server at a time.
>
> JG
>
> > -----Original Message-----
> > From: William Kang [mailto:weliam.cloud@gmail.com]
> > Sent: Monday, September 06, 2010 1:57 PM
> > To: hbase-user
> > Subject: Limits on HBase
> >
> > Hi folks,
> > I know this question may have been asked many times, but I am wondering
> > if
> > there is any update on the optimized cell size (in megabytes) and row
> > size
> > (in megabytes)? Many thanks.
> >
> >
> > William
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message