Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (nike.apache.org: 209.85.214.180 is neither permitted
 nor denied by domain of mellery@opendns.com)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1283)
Subject: Re: column count guidelines
From: Michael Ellery <mellery@opendns.com>
In-Reply-To: 
 <CALte62xnh0382v_ZaHskHK9uMWsqRkn6VYV3t5JMwYRWwrYvWw@mail.gmail.com>
Date: Thu, 7 Feb 2013 20:34:26 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <EEFF4425-7CAA-4CCC-B533-233EAB2E00ED@opendns.com>
References: <58F7FFE4-A3CE-4069-8FAD-32067CCA80B9@opendns.com>
 <CALte62xqcomTXKuqdwj8W7M5K_57QwMehGztbS06BaK-iJ7TWg@mail.gmail.com>
 <0E011DB5-3BCF-475B-8585-FB8C3ED3122E@opendns.com>
 <CALte62xnh0382v_ZaHskHK9uMWsqRkn6VYV3t5JMwYRWwrYvWw@mail.gmail.com>
To: user@hbase.apache.org


thanks for reminding me of the HBASE version in CDH4 - that's something =
we'll definitely take into consideration.

-Mike

On Feb 7, 2013, at 5:09 PM, Ted Yu wrote:

> Thanks Michael for this information.
>=20
> FYI CDH4 (as of now) is based on HBase 0.92.x which doesn't have the =
two
> features I cited below.
>=20
> On Thu, Feb 7, 2013 at 5:02 PM, Michael Ellery <mellery@opendns.com> =
wrote:
>=20
>> There is only one CF in this schema.
>>=20
>> Yes, we are looking at upgrading to CDH4, but it is not trivial since =
we
>> cannot have cluster downtime. Our current upgrade plans involves =
additional
>> hardware with side-by side clusters until everything is =
exported/imported.
>>=20
>> Thanks,
>> Mike
>>=20
>> On Feb 7, 2013, at 4:34 PM, Ted Yu wrote:
>>=20
>>> How many column families are involved ?
>>>=20
>>> Have you considered upgrading to 0.94.4 where you would be able to
>> benefit
>>> from lazy seek, Data Block Encoding, etc ?
>>>=20
>>> Thanks
>>>=20
>>> On Thu, Feb 7, 2013 at 3:47 PM, Michael Ellery <mellery@opendns.com>
>> wrote:
>>>=20
>>>> I'm looking for some advice about per row CQ (column qualifier) =
count
>>>> guidelines. Our current schema design means we have a HIGHLY =
variable CQ
>>>> count per row -- some rows have one or two CQs and some rows have
>> upwards
>>>> of 1 million. Each CQ is on the order of 100 bytes (for round =
numbers)
>> and
>>>> the cell values are null.  We see highly variable and too often
>>>> unacceptable read performance using this schema.  I don't know for =
a
>> fact
>>>> that the CQ count variability is the source of our problems, but I =
am
>>>> suspicious.
>>>>=20
>>>> I'm curious about others' experience with CQ counts per row -- are =
there
>>>> some best practices/guidelines about how to optimally size the =
number of
>>>> CQs per row. The other obvious solution will involve breaking this =
data
>>>> into finer grained rows, which means shifting from GETs to SCANs - =
are
>>>> there performance trade-offs in such a change?
>>>>=20
>>>> We are currently using CDH3u4, if that is relevant. All of our =
loading
>> is
>>>> done via HFILE loading (bulk), so we have not had to tune write
>> performance
>>>> beyond using bulk loads. Any advice appreciated, including what =
metrics
>> we
>>>> should be looking at to further diagnose our read performance
>> challenges.
>>>>=20
>>>> Thanks,
>>>> Mike Ellery
>>=20
>>=20