Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-RocketYMMF:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=ghqDg2r1y50HS7osKYLNoNYjmiqlf67ORkkZ8lQ7HC43M74Ed+oLOAFG0VBQmwg8kzL+CnBn6itGeAKBv04xAPS7z1eN/C0hUsGR8uoH1J68sHA/Un1khU+EJXgSLGSAeVBrSb3BVKeUEru/nY/Admgopeyde40mDdPL6Bz2mO4=;
References: 
 <CAPp-OAtEvqDVNvGidO1Z-BGisiDNtHWAEOTf4yJmQkkix3LBVA@mail.gmail.com>
 <CALte62zTDp2hcpOwwXZFq1rAQtH2R=m_GYu3m+VXONHgRv9yAg@mail.gmail.com>
 <CAPp-OAso+B-ViRdrBNtBEpCF=zip=rz=Z-KRV67wrNper7UEiw@mail.gmail.com>
Message-ID: <1360462639.27693.YahooMailNeo@web140605.mail.bf1.yahoo.com>
Date: Sat, 9 Feb 2013 18:17:19 -0800 (PST)
From: lars hofhansl <larsh@apache.org>
Reply-To: lars hofhansl <larsh@apache.org>
Subject: Re: Inconsistent row count between mapreduce and shell count
To: "user@hbase.apache.org" <user@hbase.apache.org>
In-Reply-To: 
 <CAPp-OAso+B-ViRdrBNtBEpCF=zip=rz=Z-KRV67wrNper7UEiw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Hmm... Can you show us the exact commands you executed?=0A=0AAnd just to ru=
le out the obvious:=0A1. There were no writes while you did the row count?=
=0A2. In the RowCount M/R case you specified neither a range nor any column=
s?=0A=0A=0ADo you always get the exact same numbers in both cases? Or do th=
ey vary?=0A=0AThanks.=0A=0A-- Lars=0A=0A=0A----- Original Message -----=0AF=
rom: kiran chitturi <chitturikiran15@gmail.com>=0ATo: user <user@hbase.apac=
he.org>=0ACc: =0ASent: Saturday, February 9, 2013 4:49 PM=0ASubject: Re: In=
consistent row count between mapreduce and shell count=0A=0AYes. I just cou=
nted the number of regions in '=0Ahttp://machine1:60010/table.jsp?name=3Ddo=
cuments' and the count is 53 which=0Ais equal to the number of complete tas=
ks in hadoop.=0A=0A=0AThanks,=0AKiran.=0A=0A=0AOn Sat, Feb 9, 2013 at 7:43 =
PM, Ted Yu <yuzhihong@gmail.com> wrote:=0A=0A> Apart from the 5 killed task=
s, was the number of successful tasks equal to=0A> the number of regions in=
 your table ?=0A>=0A> Thanks=0A>=0A> On Sat, Feb 9, 2013 at 4:14 PM, kiran =
chitturi <chitturikiran15@gmail.com=0A> >wrote:=0A>=0A> > Hi!=0A> >=0A> > I=
 am using Hbase 0.94.1 version over a distributed cluster of 20 nodes.=0A> =
>=0A> > When i execute hbase count over a table in a shell, i got the count=
 of=0A> > 2152416 rows.=0A> >=0A> > When i did the same thing using the row=
counter mapreduce, i got the value=0A> > as below=0A> >=0A> > org.apache.ha=
doop.hbase.mapreduce.RowCounter$RowCounterMapper$Counters=0A> > 13/02/10 00=
:05:06 INFO mapred.JobClient:=A0 =A0  ROWS=3D1389991=0A> >=0A> > Same thing=
 happened when i used pig to count or do operations. There is=0A> > inconsi=
stency between both the results.=0A> >=0A> > During the mapreduce, i have n=
oticed that there are 5 tasks that are=0A> > killed. When i tried to trace =
back to the tasktracker logs of the node it=0A> > shows similar to below lo=
g.=0A> >=0A> > 2013-02-09_23:58:58.40665 13/02/09 23:58:58 INFO mapred.Task=
Tracker: JVM=0A> > with ID: jvm_201302090035_0015_m_1905604998 given task:=
=0A> > attempt_201302090035_0015_m_000012_1=0A> > 2013-02-09_23:59:03.57016=
 13/02/09 23:59:03 INFO mapred.TaskTracker:=0A> > Received KillTaskAction f=
or task: attempt_201302090035_0015_m_000012_1=0A> > 2013-02-09_23:59:03.570=
34 13/02/09 23:59:03 INFO mapred.TaskTracker:=0A> About=0A> > to purge task=
: attempt_201302090035_0015_m_000012_1=0A> > 2013-02-09_23:59:03.61003 13/0=
2/09 23:59:03 INFO util.ProcessTree:=0A> Killing=0A> > process group9745 wi=
th signal TERM. Exit code 0=0A> >=0A> > I have also tried to run the tool '=
hbck' but it shows no inconsistencies.=0A> >=0A> > Can you please suggest m=
e why there is inconsistency and how can i=0A> correct=0A> > it ?=0A> >=0A>=
 > Thanks,=0A> > --=0A> > Kiran Chitturi=0A> >=0A>=0A=0A=0A=0A-- =0AKiran C=
hitturi=0A