hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: qualifier's number limitations (hbase 0.90.x)
Date Tue, 01 Mar 2011 22:52:27 GMT
>> If you want to be able to get only a few qualifiers from a scan, then
putting them in one cell will force the server to send you all of the
values, using up a lot more network I/O.

See https://issues.apache.org/jira/browse/HBASE-1537

On Tue, Mar 1, 2011 at 2:49 PM, Buttler, David <buttler1@llnl.gov> wrote:

> Everything in HBase is stored as a byte[]: table name, column name,
> qualifier name, and values.
>
> If you have your own scheme for batching a large number of qualifiers into
> a single column, then you should be able to get greater efficiency as every
> cell in hbase has a copy of the row key, column, qualifier.
>
> The limits that you will run into is total row size.  It is not a good idea
> to have a single row fill an entire region (256MB by default).  But, if you
> can keep your cells under 10 MB in your scheme, I wouldn't expect there to
> be a problem.
>
> In terms of scanning efficiency, I think it depends on how you scan your
> table.  If you scans need to return every value, then you should be ahead by
> storing them all in one column.  If you want to be able to get only a few
> qualifiers from a scan, then putting them in one cell will force the server
> to send you all of the values, using up a lot more network I/O.
>
> Dave
>
> -----Original Message-----
> From: Oleg Ruchovets [mailto:oruchovets@gmail.com]
> Sent: Tuesday, March 01, 2011 2:11 PM
> To: user@hbase.apache.org
> Subject: qualifier's number limitations (hbase 0.90.x)
>
> Hi ,
>   we have a hbase table which currently has qualifier's value a  byte[]
> array.
> We did it for the reducing qualifiers number.
>
> In case object has 5-10 fields  ,
>
>  so instead of
> qualifier1:value1
> qualifier2:value2
> qualifier3:value3
> qualifier4:value4
> qualifier5:value5
> ......
> qualifier10:value10
>
> we have
> qualifier1: byte[] array.
>
> *questions:*
> 1) I believe , using  the first approach (qualifier1:value1   and values
> are
> primitive types like int , double ) server size scans will  work much more
> faster ,  Am I write?
> 2) In our case with the first approach   we can get 50000-100000
> qualifiers.
> Is it possible to handle such number of qualifiers? What is the limitation
> of qualifiers number?
>
> Thanks in advance
> Oleg.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message