hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject Re: one column family but lots of tables
Date Sat, 24 Aug 2013 15:47:22 GMT
thanks i think it makes sense. i will go through hbase architecture again
to make sure i fully understand the mapping to "regions" and "stores"


On Fri, Aug 23, 2013 at 10:33 AM, Michael Segel
<michael_segel@hotmail.com>wrote:

> I think the issue which a lot of people miss is why do you want to use a
> column family in the first place.
>
> Column families are part of the same table structure, and each family is
> kept separate.
>
> So in your design, do you have tables which are related, but are not
> always used at the same time?
>
> The example that I use when I teach about HBase or do a
> lecture/presentation is an Order Entry system.
> Here you have an order being entered, then you have  one or many pick
> slips being generated, same for shipping then there's the invoicing.
> All separate processes which relate back to the same order.
>
> So here it makes sense to use column families.
>
> Other areas could be metadata is surrounding a transaction. Again... few
> column families are tied together.
>
> Does this make sense?
>
>
> On Aug 23, 2013, at 12:35 AM, lars hofhansl <larsh@apache.org> wrote:
>
> > You can think of it this way: Every region and column family is a
> "store" in HBase. Each store has a memstore and its own set of HFiles in
> HDFS.
> > The more stores you have, the more there is to manage.
> >
> > So you want to limit the number of stores. Also note that the word
> "Table" is somewhat a misnomer in HBase it should have better been called
> "Keyspace".
> > The extra caution for the number of column families per table stems from
> the fact that HBase flushes by region (which means all stores of that
> region are flushed). This in turn means that unless are column families
> hold roughly the same amount of data you end up with very lopsided
> distributions of HFile sizes.
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> > From: Koert Kuipers <koert@tresata.com>
> > To: user@hbase.apache.org; vrodionov@carrieriq.com
> > Sent: Thursday, August 22, 2013 12:30 PM
> > Subject: Re: one column family but lots of tables
> >
> >
> > if that is the case, how come people keep warning about limiting the
> number
> > of column families to only a handful (with more hbase performance will
> > degrade supposedly), yet there seems to be no similar warnings for number
> > of tables? see for example here:
> > http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/27616
> >
> > if a table means at least one column family then the number of tables
> > should also be kept to a minumum, no?
> >
> >
> >
> >
> > On Thu, Aug 22, 2013 at 1:58 PM, Vladimir Rodionov
> > <vrodionov@carrieriq.com>wrote:
> >
> >> Nope. Column family is per table (its sub-directory inside your table
> >> directory in HDFS).
> >> If you have N tables you will always have , at least, N distinct CFs
> (even
> >> if they have the same name).
> >>
> >> Best regards,
> >> Vladimir Rodionov
> >> Principal Platform Engineer
> >> Carrier IQ, www.carrieriq.com
> >> e-mail: vrodionov@carrieriq.com
> >>
> >> ________________________________________
> >> From: Koert Kuipers [koert@tresata.com]
> >> Sent: Thursday, August 22, 2013 8:06 AM
> >> To: user@hbase.apache.org
> >> Subject: one column family but lots of tables
> >>
> >> i read in multiple places that i should try to limit the number of
> column
> >> families in hbase.
> >>
> >> do i understand it correctly that when i create lots of tables, but they
> >> all use the same column family (by name), that i am just using one
> column
> >> family and i am OK with respect to limiting number of column families ?
> >>
> >> thanks! koert
> >>
> >> Confidentiality Notice:  The information contained in this message,
> >> including any attachments hereto, may be confidential and is intended
> to be
> >> read only by the individual or entity to whom this message is
> addressed. If
> >> the reader of this message is not the intended recipient or an agent or
> >> designee of the intended recipient, please note that any review, use,
> >> disclosure or distribution of this message or its attachments, in any
> form,
> >> is strictly prohibited.  If you have received this message in error,
> please
> >> immediately notify the sender and/or Notifications@carrieriq.com and
> >> delete or destroy any copy of this message and its attachments.
>
> The opinions expressed here are mine, while they may reflect a cognitive
> thought, that is purely accidental.
> Use at your own risk.
> Michael Segel
> michael_segel (AT) hotmail.com
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message