Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of kelvin.msj@gmail.com designates
 209.85.219.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <CC776550.4D14F%doug.meil@explorysmedical.com>
References: 
 <CANDVYVkdOm5=B8bnAwm13A-T-DNrSxquxq_LGfMVxt=j3=G2GQ@mail.gmail.com>
	<CC776550.4D14F%doug.meil@explorysmedical.com>
Date: Thu, 13 Sep 2012 15:35:25 +0100
Message-ID: 
 <CANDVYV=wGC_bePca3Vg4SUg6+M2R5q8ehP+TCQAbKdOc9mtYHA@mail.gmail.com>
Subject: Re: Hbase Scan - number of columns make the query performance way
 different
From: Shengjie Min <kelvin.msj@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=e89a8f6436f8d1d1ef04c996375b

--e89a8f6436f8d1d1ef04c996375b
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

In my case, I am not feeding hbase result to mapred, it's just pure hbase
scan, returning all columns vs two columns makes huge difference to me.

On 13 September 2012 15:29, Doug Meil <doug.meil@explorysmedical.com> wrote=
:

>
> Hi there, I don't know the specifics of your environment, but ...
>
> http://hbase.apache.org/book.html#perf.reading
> 11.8.2. Scan Attribute Selection
>
>
> =C5=A0 describes paying attention to the number of columns you are return=
ing,
> particularly when using HBase as a MR source.  In short, returning only
> the columns you need means you are reducing the data transferred between
> the RS and the client and the number of KV's evaluated in the RS, etc.
>
>
>
>
> On 9/13/12 10:12 AM, "Shengjie Min" <kelvin.msj@gmail.com> wrote:
>
> >Hi,
> >
> >I found an interesting difference between hbase scan query.
> >
> >I have a hbase table which has a lot of columns in a single column famil=
y.
> >eg. let's say I have a users table, then userid, username, email .... et=
c
> >etc 15 fields all together are in the single columnFamily.
> >
> >if you are familiar with RDBMS,
> >
> >query 1: select * from users
> >vs
> >query 2: select userid, username from users
> >
> >in mysql, these two has a difference, the query 2 will be obviously
> >faster,
> >but two queries won't give you a huge difference from performance
> >perspective.
> >
> >In Hbase, I noticed that:
> >
> >query 3: scan 'users',   // this is basically return me all 15 fields
> >vs
> >query 4: scan 'users', {COLUMNS=3D>['cf:userid','cf:username']}    // th=
is
> >is
> >return me only two fields: userid , username
> >
> >query 3 here takes way longer than query 4, Given a big data set. In my
> >test, I have around 1,000,000 user records. You are talking about query =
3
> >-
> >100 secs VS query 4 - a few secs.
> >
> >
> >Can anybody explain to me, why the width of the resultset in HBASE can
> >impact the performance that much?
> >
> >
> >Shengjie Min
>
>
>


--=20
All the best,
Shengjie Min

--e89a8f6436f8d1d1ef04c996375b--