hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Corgan <mcor...@hotpads.com>
Subject Re: HBase Developer's Pow-wow.
Date Tue, 11 Sep 2012 05:59:43 GMT
Jacques - i'll be there tomorrow.  Look forward to talking.  Some comments
before then:

- How to maintain consistency (maybe this is unimportant?)

Not unimportant at all.  In fact, I picture the whole secondary index
conversation as a lower level goal of supporting consistent cross-region
updates.  I'm hesitant on some of the region co-location ideas because they
look like optimizations on the end goal of consistency across servers.  All
of the optimizations are nice, but the real meat of the problem is how to
bake cross-region consistency in at the ground level as opposed to patching
it on as the failure case where an index region gets separated from its
parent.  It would be better to get the cross-server stuff working first,
and then optimize the same-server scenario.  That is, like you say, if
anybody has time =)

- How to avoid network bottleneck as the cluster expands (in the case of
> a per-table approach, you're going to have pass primary keys
> around constantly except in the case that the only value you want is the
> indexed value and you saved that entire value in the index table.)

In my use cases, i typically scan batches of ~1000 index entries from the
index table (~1 RPC / ~1 data block), and then i issue a multiGet to fetch
the primary rows.  Because the index is sorted by the primary rows, they
all go to the first region in the table which again equates to ~1 RPC.  So
maybe it's 2 RPC's instead of 1 which doesn't seem too bad.

- How to maximize scale.  (In the per table case, a particular set of indexed
> values will probably be colocated among a fraction of all nodes.

Writes will definitely be slightly faster in the per-region case, but at
the huge expense of reads having to go to multiple servers.  In terms of
number of regions (R), the additional write expense is O(1) whereas the
read expense is on average O(R/2).  If you have 100 regions of users and
want to look up a userId by email, you have to jump through 50 regions on
average to find the user.

I spent some time in the Cassandra community doing a review of various indexing
> use cases.  I should go take another look to see what they do and how it
> works for them...

HBase has a lot of similarities to Cassandra but i would say it is a
different beast when it comes to indexing.  The biggest difference (even
bigger than the tunable consistency) is the fact that hbase stores all rows
in a sorted order that automatically split into regions and evenly
distributed.  Cassandra is not designed to host unpredictably growing
sorted tables (like secondary index tables tend to be), so it makes some
concessions in index design.  Instead of storing each index entry as a
separate row in a rapidly growing table, which hbase deals with nicely
because it can split/balance the index table, cassandra stores all of the
index entries for an index value as columns (qualifiers) in the same row.
 For low cardinality indexes this can create several huge rows which become
hotspots.  Said differently, cassandra is forced to create indexes using
wide tables, where hbase has the luxury of using tall tables.  My cassandra
knowledge is dated, so please correct me if that's wrong.

On Mon, Sep 10, 2012 at 9:47 PM, Ramkrishna.S.Vasudevan <
ramkrishna.vasudevan@huawei.com> wrote:

> Hi
> Yes, a separate index table along with the main table and the master should
> ensure that the regions of both tables are collocated during assignments.
> The regions in index table can be same as that of the main table in the
> sense that both should have the same start and endkeys.
> Different indices can be grouped within these regions.
> In case of spare data definitely the index creation is going to be a
> beneficial one.
> In case of dense data may be the indices may be an overhead in some cases.
> In one of the wiki pages of Cassandra I also read that they suggest to have
> atleast one EQUALS condition in the query that tries to use indices. This
> will help in confining the results to a specific set and over which the
> range queries can be applied.  So may be at the first level we can see what
> gain we get when we use EQUALs condition but any way the framework can be
> generic to handle range queries and EQUALs condition queries.
> After the meet up is over, I can go through the discussion topics and
> provide our experiences also.
> Regards
> Ram
> > -----Original Message-----
> > From: Andrew Purtell [mailto:apurtell@apache.org]
> > Sent: Tuesday, September 11, 2012 9:52 AM
> > To: dev@hbase.apache.org
> > Subject: Re: HBase Developer's Pow-wow.
> >
> > Regarding this:
> >
> > On Mon, Sep 10, 2012 at 12:13 PM, Matt Corgan <mcorgan@hotpads.com>
> > wrote:
> > > 1) Per-region or Per-table
> > [...]
> > > 1)
> > > - Per-region: the index entries are stored on the same machine as the
> > > primary rows
> > > - Per-table: each index is stored in a separate table, requiring
> > > cross-server consistency
> >
> > LarsH and I were discussing this a bit. This doesn't have to be a
> > choice, it could be possible to have both, a separate table for index
> > storage, and colocation of the index table regions and primary table
> > regions on the same regionserver so cross-region consistency issues
> > can be dealt with through low latency in-memory channels. (With
> > fallback to cross-server consistency mechanism when placement can't be
> > ideal when the cluster is out of steady state due to failure/churn.)
> > The master might assign primary and index regions out together as a
> > group.
> >
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein (via Tom White)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message