Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of
 ramkrishna.s.vasudevan@gmail.com designates 209.85.215.176 as permitted
 sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAOtJ30qiHBL1S8Q0V0c4pOn3FX2H_4i7E5Y9OHJVsH4+YaimNA@mail.gmail.com>
References: <609644380836277084@unknownmsgid>
	<CAOtJ30qiHBL1S8Q0V0c4pOn3FX2H_4i7E5Y9OHJVsH4+YaimNA@mail.gmail.com>
Date: Thu, 17 Jan 2013 22:39:45 +0530
Message-ID: 
 <CAAT7MkrybEF_gHCaeG6rrhLKr534sbvNhQbHtdLHxsb8s8os2A@mail.gmail.com>
Subject: Re: Loading data, hbase slower than Hive?
From: ramkrishna vasudevan <ramkrishna.s.vasudevan@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=047d7b5dbe20c9fb3104d37f0f71

--047d7b5dbe20c9fb3104d37f0f71
Content-Type: text/plain; charset=ISO-8859-1

Hive is more for batch and HBase is for more of real time data.

Regards
Ram

On Thu, Jan 17, 2013 at 10:30 PM, Anoop John <anoop.hbase@gmail.com> wrote:

> In case of Hive data insertion means placing the file under table path in
> HDFS.  HBase need to read the data and convert it into its format. (HFiles)
> MR is doing this work..  So this makes it clear that HBase will be slower.
> :)  As Michael said the read operation...
>
>
>
> -Anoop-
>
> On Thu, Jan 17, 2013 at 10:14 PM, Austin Chungath <austincv@gmail.com
> >wrote:
>
> >   Hi,
> > Problem: hive took 6 mins to load a data set, hbase took 1 hr 14 mins.
> > It's a 20 gb data set approx 230 million records. The data is in hdfs,
> > single text file. The cluster is 11 nodes, 8 cores.
> >
> > I loaded this in hive, partitioned by date and bucketed into 32 and
> sorted.
> > Time taken is 6 mins.
> >
> > I loaded the same data into hbase, in the same cluster by writing a map
> > reduce code. It took 1hr 14 mins. The cluster wasn't running anything
> else
> > and assuming that the code that i wrote is good enough, what is it that
> > makes hbase slower than hive in loading the data?
> >
> > Thanks,
> > Austin
> >
>

--047d7b5dbe20c9fb3104d37f0f71--