Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of mundra@gmail.com designates
 209.85.214.43 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CABFz-zj42ZpuBGfn=Rqy=esHPgfO=XFmep+h7=Uis2PAgEod_w@mail.gmail.com>
References: 
 <CAMpwW3q2+LF+JRSAz=f+ygf+D5229iT+iSXQC2d73VRP88gDng@mail.gmail.com>
	<CABFz-zj42ZpuBGfn=Rqy=esHPgfO=XFmep+h7=Uis2PAgEod_w@mail.gmail.com>
Date: Fri, 25 Jan 2013 23:29:22 +0530
Message-ID: 
 <CAMpwW3rfrNxmsnzZDa6wdqNbNJa5O9rUXEpGx2rspj8orFGpaw@mail.gmail.com>
Subject: Re: Hbase scans taking a lot of time
From: Vibhav Mundra <mundra@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=00151747bd0ceb75a004d420afe2

--00151747bd0ceb75a004d420afe2
Content-Type: text/plain; charset=ISO-8859-1

The number of column families I have is 13, which I guess is okie?

-Vibhav


On Fri, Jan 25, 2013 at 11:01 PM, Luke Lu <llu@apache.org> wrote:

> You'll have this problem if you have a large number of column families
> being scanned/populated at the same time. Make sure the data you
> scan/populate frequently are in the same column family (you can have many
> columns in a column family). Unlike BigTable/Hypertable which has the
> concept of locality/access groups, HBase always stores column families in
> separate files, essentially making column family not only a logic grouping
> mechanism but also a physical locality group.
>
>
> On Fri, Jan 25, 2013 at 1:10 AM, Vibhav Mundra <mundra@gmail.com> wrote:
>
> > I am facing a very strange problem with HBase.
> >
> > This what I did:
> > a) Create a table, using pre partioned splits.
> > b) Also the column familes are zipped with lzo compression.
> > c) Using the above configuration I am able to populate 2 million row per
> > min in the Hbase.
> > d) I have created a table with 300 million odd rows, which roughy took
> me 3
> > hours to populate and the data size is of 25GB.
> >
> > e) But when I query for data the performance I am getting is very bad.
> >    Basically this is what I am seeing: High CPU, no disk I/O and network
> > I/O is happening at the rate of 6~7MB secs.
> >
> >
> > Because of this, if I scan the entries of the table using Hive it is
> taking
> > ages.
> > Basically it is taking around 24 hours to scan the table. Any idea, of
> how
> > to debug.
> >
> >
> > -Vibhav
> >
>

--00151747bd0ceb75a004d420afe2--