Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of vashishtha.h@gmail.com
 designates 209.85.214.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=bNlFGDCOfj+gqZYaFKuJR0NcS6LpKrLiQbNH/4LbTC23dWpqIc8lwOgBwS1DEr4mXU
         fqt0GC5T2ooyUV71p9wCazgkxvqN8nqkja1AdMSrrKe1bo2tp+RSiZEDuETWXUudwMbU
         I3hdb4YfBESCg3elAujKlwPBZDNw2dbNSVQDo=
MIME-Version: 1.0
In-Reply-To: <AANLkTimUHORBpUgqBa9P5FdjTJ5evLVO-EYdCP5fECVK@mail.gmail.com>
References: <AANLkTi=LT2TaL_yFwP-gs1zLca=CbvoR=BQc4LpR1x9_@mail.gmail.com>
	<5A76F6CE309AD049AAF9A039A3924282073D5B36@sc-mbx04.TheFacebook.com>
	<AANLkTimUHORBpUgqBa9P5FdjTJ5evLVO-EYdCP5fECVK@mail.gmail.com>
Date: Tue, 7 Sep 2010 00:49:41 -0600
Message-ID: <AANLkTimcT7YFz57DxiOmvVJd-QWycST7dHwDV8fyepsT@mail.gmail.com>
Subject: Re: Limits on HBase
From: Himanshu Vashishtha <vashishtha.h@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=0022150490db35f0ce048fa5cd51

--0022150490db35f0ce048fa5cd51
Content-Type: text/plain; charset=ISO-8859-1

Assuming you will be using hdfs as the file system: wouldn't saving those
large objects in the fs and keeping a pointer to them in a hbase table serve
the purpose.

[I haven't done it myself but I can't see it not working. In fact, I
remember reading it somewhere in the list.]

~Himanshu

On Mon, Sep 6, 2010 at 11:40 PM, William Kang <weliam.cloud@gmail.com>wrote:

> Hi JG,
> Thanks for your reply. As far as I have read in Hbase's documentation and
> wiki, the cell size is not supposed to be larger than 10 MB. For the row, I
> am not quite sure, but it looks like 256 MB is the upper limit. I am
> considering store some binary data used to be stored in RDBM blob field.
> The
> size of those binary objects may vary from hundreds of KB to hundreds of
> MB.
> What would be a good way to use Hbase for it? We really want to use hbase
> to
> avoid that scaling problem.
> Many thanks.
>
>
> William
>
> On Mon, Sep 6, 2010 at 7:10 PM, Jonathan Gray <jgray@facebook.com> wrote:
>
> > I'm not sure what you mean by "optimized cell size" or whether you're
> just
> > asking about practical limits?
> >
> > HBase is generally used with cells in the range of tens of bytes to
> > hundreds of kilobytes.  However, I have used it with cells that are
> several
> > megabytes, up to about 50MB.  Up at that level, I have seen some weird
> > performance issues.
> >
> > The most important thing is to be sure to tweak all of your settings.  If
> > you have 20MB cells, you need to be sure to increase the flush size
> beyond
> > 64MB and the split size beyond 256MB.  You also need enough memory to
> > support all this large object allocation.
> >
> > And of course, test test test.  That's the easiest way to see if what you
> > want to do will work :)
> >
> > When you run into problems, e-mail the list.
> >
> > As far as row size is concerned, the only issue is that a row can never
> > span multiple regions so a given row can only be in one region and thus
> be
> > hosted on one server at a time.
> >
> > JG
> >
> > > -----Original Message-----
> > > From: William Kang [mailto:weliam.cloud@gmail.com]
> > > Sent: Monday, September 06, 2010 1:57 PM
> > > To: hbase-user
> > > Subject: Limits on HBase
> > >
> > > Hi folks,
> > > I know this question may have been asked many times, but I am wondering
> > > if
> > > there is any update on the optimized cell size (in megabytes) and row
> > > size
> > > (in megabytes)? Many thanks.
> > >
> > >
> > > William
> >
>

--0022150490db35f0ce048fa5cd51--