Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.co.in;
  h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=i+y1qRtlSRAacRcXd0gFE7fb3aR+dOfPeOMGLs7SDIYGCsW9+EEw9B6q0ifRZNDzk8NadK0rNUd91MvfaW9nHmKN5x1tkF/1/Nmlly+LY1MXG9aL6VPV4sEpBHoIcBqb57l0/fHb4RzpHhCR8dYIfOiia+lWNgt/HRf6NnSqsG4=;
References: 
 <1357493340.78254.BPMail_high_noncarrier@web192901.mail.sg3.yahoo.com>
 <CAE24rAd=zTTVugNBdoH4gFvf+9yYBb5BhJ6u9nNBrjbSf+MhjA@mail.gmail.com>
Message-ID: <1358896243.60271.YahooMailNeo@web192906.mail.sg3.yahoo.com>
Date: Wed, 23 Jan 2013 07:10:43 +0800 (SGT)
From: Dhaval Shah <prince_mithibai@yahoo.co.in>
Reply-To: Dhaval Shah <prince_mithibai@yahoo.co.in>
Subject: Re: Controlling TableMapReduceUtil table split points
To: "user@hbase.apache.org" <user@hbase.apache.org>
In-Reply-To: 
 <CAE24rAd=zTTVugNBdoH4gFvf+9yYBb5BhJ6u9nNBrjbSf+MhjA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="-1395031902-291880844-1358896243=:60271"

---1395031902-291880844-1358896243=:60271
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

=0AHi David.. We successfully use the "logical" schema approach and have no=
t seen issues yet.. Ofcourse it all depends on the use case and saying it w=
ould work for you because it works for us would be naive.. However, if it d=
oes work, it will make your life much easier because with a logical schema =
other problems become simpler (like you can be sure that 1 map function wil=
l process an entire row rather than a row going to multiple mappers, or if =
you are using filters that restrict queries to only a small subset of the d=
ata, even setBatch won't be needed for those use cases).. I did run into is=
sues where I did not use setBatch and my mappers ran out of memory but that=
 was a simpler one to solve (and by the way if you are on CDH4, the HBase e=
xport utility also does not use setBatch and your mapper will run out of me=
mory if you have a large row.. Its easy to put that line in though as a con=
fig param and this feature is available in future releases of HBase
 trunk)=0A=0ARegards,=0ADhaval=0A =0A=0A________________________________=0A=
 From: David Koch <ogdude@googlemail.com>=0ATo: user@hbase.apache.org =0ASe=
nt: Sunday, 6 January 2013 12:53 PM=0ASubject: Re: Controlling TableMapRedu=
ceUtil table split points=0A  =0AHi Dhaval,=0A=0AGood call on the setBatch.=
 I had forgotten about it. Just like changing the=0Aschema it would involve=
 changing the map(...) to reflect the fact that only=0Apart of the user's d=
ata is returned in each call but I would not have to=0Amanipulate table spl=
its.=0A=0AThe HBase book does suggest that it's bad practice to use the "lo=
gical"=0Aschema of lumping all user data into a single row(*) but I'll do s=
ome=0Atesting to see what works.=0A=0AThank you,=0A=0A/David=0A=0A(*) Chapt=
er 9, section "Tall-Narrow Versus Flat-Wide Tables", 3rd ed., page=0A359)=
=0A=0A=0AOn Sun, Jan 6, 2013 at 6:29 PM, Dhaval Shah <prince_mithibai@yahoo=
.co.in>wrote:=0A=0A> Another option to avoid the timeout/oome issues is to =
use scan.setBatch()=0A> so that the scanner would function normally for sma=
ll rows but would break=0A> up large rows in multiple Result objects which =
you can now use in=0A> conjunction with scan.setCaching() to control how mu=
ch data you get back..=0A>=0A> This approach would not need a change in you=
r schema design and would=0A> ensure that only 1 mapper processes the entir=
e row (but in multiple calls=0A> to the map function)=0A>
---1395031902-291880844-1358896243=:60271--