hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Please help me overcome HBase's weaknesses
Date Sun, 05 Sep 2010 04:04:17 GMT
MauMau:

public void createTable(HTableDescriptor desc, byte [] startKey,
    byte [] endKey, int numRegions)

If you choose HBase 0.20.6, please be aware that you need to apply
HBASE-2473 yourself so that you can use the above API.

On Sat, Sep 4, 2010 at 6:49 PM, MauMau <maumau307@gmail.com> wrote:

> Hello, Stack,
>
> Thank you for giving me advice.
>
>
>  But your boss seems rather to be criticizing the fact that our system
>> is made of components.  In software engineering, this is usually
>> considered a strength.  As to 'roles', one of the bigtable author's
>> argues that a cluster of master and slaves makes for simpler systems
>> [1].
>>
>
> I definitely agree with you. However, my boss considers the simplicity from
> the users' viewpoint. More components make the system more complex for
> users.
>
>
>
>  If you have big data, you are running hadoop already?  If so, you
>> already have hdfs in place.  If you have big systems and are trying to
>> do HA, you'll have a zookeeper ensemble in place already or at least,
>> if you haven't, you are likely considering it?  In this case, HBase
>> complements your existing infrastructure?
>>
>
> Our potential customers will create new systems for new big data. Therefore
> they are not using Hadoop yet.
>
>
>
>  Out of the box, HBase can manage zookeeper for you.  There is an issue
>> to make it so when 'start' hbase, by default it starts hdfs too.
>> Would that help?
>>
>
> This might help, if the users do not need to be aware of
> starting/stopping/initializing HDFS and its daemons as much as possible.
>
>
>
>  I've tripped over blog posts where zookeeper is grafted to a cassandra
>> ensemble to add facility that is inherent to hbase; e.g. locks and
>> counters (though in the first case, zk is not suitable for cluster
>> locks and yes, I've also read of the patch to do counters using vector
>> clocks).
>>
>
> Yes, I know that the current Cassandra requires the users to use ZooKeeper
> to implement counters. I'm afraid that if counters with vector clocks is
> implemented, one of HBase's merits -- not needing another software just for
> counters -- will be lost.
>
>
>
>  Why can't you do the same in hbase?  Hash sensor_id + ts and have it
>> distributed across all of the cluster and have no 'hotspot'?
>>
>
> Yes, hashing the sensor_id may distributes inserts well right after
> creating a new table, by using the following HBaseAdmin method (this method
> is not documented). I'll consider how to utilize this method.
>
>  public void createTable(HTableDescriptor desc, byte [] startKey,
>     byte [] endKey, int numRegions)
>
>
>
>  At stumbleupon we have a sensor database built on hbase.  Purportedly
>> its to be made open.  Watch this list for announcements.
>>
>
> Sounds interesting. I'll look forward to that announcement.
>
> Regards,
> Maumau
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message