Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.de;
  h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=v5fHzB90B2z4ewkvvIuw+TpnlxdrtC3Dz1+ugv86OTI3n/XbIZkZP31TGnmFYRPyany6/0jrIqhJAbqEhxBzQnpe7d6pfNaK+M0g1YKPTQ37X+duWUZd4xjcuOLbnG8wpZGS/28pE5HheEGpi6GB5IeA1wtezOeBjhYOtIMWuzY=;
References: <1343748460.12346.YahooMailNeo@web171503.mail.ir2.yahoo.com>
 <CAG+ckK_tajNHuAB+G-2u3yJoSPq+YZUagFGnHC8dMsd1q7QXVg@mail.gmail.com>
 <CAOKsKJWR48Cz-P2zrec1ebQ2J14UDLgjOdYeaJwDgB_SGuN7sw@mail.gmail.com>
Message-ID: <1343809130.3275.YahooMailNeo@web171503.mail.ir2.yahoo.com>
Date: Wed, 1 Aug 2012 09:18:50 +0100 (BST)
From: =?iso-8859-1?Q?Christian_Sch=E4fer?= <syrious3000@yahoo.de>
Reply-To: =?iso-8859-1?Q?Christian_Sch=E4fer?= <syrious3000@yahoo.de>
Subject: Re: How to query by rowKey-infix
To: "user@hbase.apache.org" <user@hbase.apache.org>
In-Reply-To: 
 <CAOKsKJWR48Cz-P2zrec1ebQ2J14UDLgjOdYeaJwDgB_SGuN7sw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks Matt & Jerry for your replies.=0A=0AThe data for each row is small (=
some hundred Bytes).=0A=0ASo, I will try the parallel table scan at first a=
s you suggested...=0ABefore organizing that by myself, wouldn't it be a bet=
ter idea to create a map reduce job for that?=0A=0AI'm not so keen on imple=
menting secondary indices especially due to the mentioned consistency conce=
rns.=0AUnfortunately projects like ithbase and ihbase are no more supportin=
g current hbase and secondary indexes by coprocessors seems are not yet to =
there.=0AIf I'm wrong feel free to correct me :)=0A=0Aregards,=0AChris=0A=
=0A=0A=0A----- Urspr=FCngliche Message -----=0AVon: Matt Corgan <mcorgan@ho=
tpads.com>=0AAn: user@hbase.apache.org=0ACC: Christian Sch=E4fer <syrious30=
00@yahoo.de>=0AGesendet: 19:41 Dienstag, 31.Juli 2012=0ABetreff: Re: How to=
 query by rowKey-infix=0A=0AWhen deciding between a table scan vs secondary=
 index, you should try to=0Aestimate what percent of the underlying data bl=
ocks will be used in the=0Aquery.=A0 By default, each block is 64KB.=0A=0AI=
f each user's data is small and you are fitting multiple users per block,=
=0Athen you're going to need all the blocks, so a tablescan is better becau=
se=0Ait's simpler.=A0 If each user has 1MB+ data then you will want to pick=
 out=0Athe individual blocks relevant to each date.=A0 The secondary index =
will help=0Ayou go directly to those sparse blocks, but with a cost in comp=
lexity,=0Aconsistency, and extra denormalized data that knocks primary data=
 out of=0Ayour block cache.=0A=0AIf latency is not a concern, I would start=
 with the table scan.=A0 If that's=0Atoo slow you add the secondary index, =
and if you still need it faster you=0Ado the primary key lookups in paralle=
l as Jerry mentions.=0A=0AMatt=0A=0AOn Tue, Jul 31, 2012 at 10:10 AM, Jerry=
 Lam <chilinglam@gmail.com> wrote:=0A=0A> Hi Chris:=0A>=0A> I'm thinking ab=
out building a secondary index for primary key lookup, then=0A> query using=
 the primary keys in parallel.=0A>=0A> I'm interested to see if there is ot=
her option too.=0A>=0A> Best Regards,=0A>=0A> Jerry=0A>=0A> On Tue, Jul 31,=
 2012 at 11:27 AM, Christian Sch=E4fer <syrious3000@yahoo.de=0A> >wrote:=0A=
>=0A> > Hello there,=0A> >=0A> > I designed a row key for queries that need=
 best performance (~100 ms)=0A> > which looks like this:=0A> >=0A> > userId=
-date-sessionId=0A> >=0A> > These queries(scans) are always based on a user=
Id and sometimes=0A> > additionally on a date, too.=0A> > That's no problem=
 with the key above.=0A> >=0A> > However, another kind of queries shall be =
based on a given time range=0A> > whereas the outermost left userId is not =
given or known.=0A> > In this case I need to get all rows covering the give=
n time range with=0A> > their date to create a daily reporting.=0A> >=0A> >=
 As I can't set wildcards at the beginning of a left-based index for the=0A=
> > scan,=0A> > I only see the possibility to scan the index of the whole t=
able to=0A> collect=0A> > the=0A> > rowKeys that are inside the timerange I=
'm interested in.=0A> >=0A> > Is there a more elegant way to collect rows w=
ithin time range X?=0A> > (Unfortunately, the date attribute is not equal t=
o the timestamp that is=0A> > stored by hbase automatically.)=0A> >=0A> > C=
ould/should one maybe leverage some kind of row key caching to=0A> accelera=
te=0A> > the collection process?=0A> > Is that covered by the block cache?=
=0A> >=0A> > Thanks in advance for any advice.=0A> >=0A> > regards=0A> > Ch=
ris=0A> >=0A>=0A