Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of bradfordstephens@gmail.com
 designates 209.85.210.185 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=F162hqBODFwR+lzrCO5zgl1cK4owwjUG/w71M/vNB1cOxvIfyRWHa0uWIcGO2G0dJ0
         kLEOFCfAQWtXPWswVT6IV2Cdos6wgJefd8gxdIGxHCtZ05mQGmlB3tqkmrSBiOOV7KrZ
         BFLLbewBGxrPQXMVNm8rSXSrduYwmg1yKwRAk=
MIME-Version: 1.0
In-Reply-To: <7c962aed0909021147x2bfb6aa6jf2db976ef04ce0d6@mail.gmail.com>
References: <127484.98071.qm@web65513.mail.ac4.yahoo.com>
	 <fa03480d0909020953t4447730pd9d51a2fe5a52114@mail.gmail.com>
	 <7c962aed0909021147x2bfb6aa6jf2db976ef04ce0d6@mail.gmail.com>
Date: Wed, 2 Sep 2009 14:09:42 -0700
Message-ID: <860544ed0909021409u24a9c7f1ne97e5b79822a01d5@mail.gmail.com>
Subject: Re: Cassandra vs HBase
From: Bradford Stephens <bradfordstephens@gmail.com>
To: hbase-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hey there,

Expect a post on my blog in a few days on this subject, with
performance and feature comparisons :)

Cheers,
Bradford

On Wed, Sep 2, 2009 at 11:47 AM, stack<stack@duboce.net> wrote:
> On Wed, Sep 2, 2009 at 9:53 AM, Schubert Zhang <zsongbo@gmail.com> wrote:
>
>> Regardless Cassandra, I want to discuss some questions about
>> HBase/Bigtable. =A0Any advices are expected.
>>
>> Regards runing MapReduce to scan/analyze big data in HBase.
>>
>> Compared to sequentially reading data from HDFS files directly,
>> scan/sequential-reading data from HBase is slower. (As my test, at least
>> 3:1
>> or 4:1).
>>
>>
> Is it really 3 to 4 times slower? =A0My guess is that it varies with data
> sizes but, a while back I compared hbase reading and raw reading from
> mapfiles with no hbase in between. =A0True, I was seeing that mapfiles we=
re
> almost 3x faster when scanning but for other dimensions, hbase was close =
to
> raw mapfile speeds.
>
> We have some ideas for improving on our current speeds (e.g. we use dfs
> pread getting blocks from hdfs always -- should switch from pread when we
> figure access is scan). =A0These are in the works.
>
>
>
>> For the data in HBase, it is diffcult to only analyze specified part of
>> data. For example, it is diffcult to only analyze the recent one day of
>> data. In my application, I am considering partition data into different
>> HBase tables (e.g. one day - one table), then, I can only touch one tabl=
e
>> for analyze via MapReduce.
>> In Google's Bigtable paper, in the "8.1 Google Analytics", they also
>> discribe this usage, but I don't know how.
>>
>
> Time-keyed row keys are a bit tough. =A0What about adding to the tail of =
a
> continuing table or does the data come in to fast? =A0If you could add to=
 the
> end of your table, =A0you MR against the tail only? =A0Can you use the ve=
rsion
> dimension of hbase? =A0Would be full table-scan but would be all server-s=
ide
> so should be fast.
>
>
>
>>
>> It is also slower to put flooding data into HBase table than writing to
>> files. (As my test, at least 3:1 or 4:1 too). So, maybe in the future,
>> HBase
>> can provide a bulk-load feature, like PNUTS?
>>
>>
> In my tests -- see up in the wiki -- for sequential write, was less than =
2x
> (can't random write into mapfile).
>
> A first cut exists in hbase-48. =A0There is a reducer which sorts all key=
s on
> a row and an hfile output format that writes a single file into a region.
> Absent is the necessary partitioner to ensure global sort order. =A0A gen=
eric
> sorter is not possible since the partitioner needs to have knowledge of y=
our
> key space. =A0Try it and let us know. =A0Currently it works populating a =
new
> table only. =A0Shouldn't be hard to rig it to populate an extant table bu=
t I'm
> not going to work on it unless there is interest by others.
>
>
> Many a time, I am thinking, maybe we need a data storage engine, which ne=
ed
>> not so strong consistency, and it can provide better writing and
>> reading throughput like HDFS. Maybe, we can design another system like a
>> simpler HBase ?
>>
>> You think its the consistency that costs? =A0HBase is a pretty
> straightforward system as is. =A0How would you simplify Schubert? =A0We c=
an work
> on improving the performance to cut down on those 4X and 3Xs that you are
> seeing. =A0A schema for time-series is a bit tough though if you want to =
key
> it by timestamp. =A0You could try Cassandra and let it hash your keys so =
they
> got distributed around the cluster but my guess is that the scan would be
> slow if you needed to access the content ordered?
>
> Thanks,
> St.Ack
>


--=20
http://www.roadtofailure.com -- The Fringes of Scalability, Social
Media, and Computer Science