hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hansi Klose" <hansi.kl...@web.de>
Subject Aw: Re: replication verifyrep
Date Tue, 15 Apr 2014 07:17:50 GMT
Hi Jean-Daniel,

thank you for your answer and bring some light into the darkness.

> You can see the bad rows listed in the user logs for your MR job.

What log do you mean. The output from the command line? 
I only see the count of GOOD or BAD rows.
Are the bad rows listed in that log which are not replicated?

Regards Hansi

> Gesendet: Montag, 14. April 2014 um 19:25 Uhr
> Von: "Jean-Daniel Cryans" <jdcryans@apache.org>
> An: "user@hbase.apache.org" <user@hbase.apache.org>
> Betreff: Re: replication verifyrep
>
> Yeah you should use endtime, it was fixed as part of
> https://issues.apache.org/jira/browse/HBASE-10395.
> 
> You can see the bad rows listed in the user logs for your MR job.
> 
> J-D
> 
> 
> On Mon, Apr 14, 2014 at 3:06 AM, Hansi Klose <hansi.klose@web.de> wrote:
> 
> > Hi,
> >
> > I wrote a little script which should control the running replication.
> >
> > The script is triggered by cron and executes the following command with
> > the actual time stamp in "endtime" and
> > a time stamp => endtime - 10800000 milli seconds. So the time frame is 3
> > hours.
> >
> > hadoop jar /usr/lib/hbase/hbase.jar verifyrep --starttime=1397217601927
> > --endtime=1397228401927 --families=t 1 tablename 2>&1
> >
> > After some running's the script found some BADROWS.
> >
> > 14/04/11 17:04:05 INFO mapred.JobClient:     BADROWS=176
> > 14/04/11 17:04:05 INFO mapred.JobClient:     GOODROWS=2
> >
> > I executed the same command 20 Minutes later in the shell and got :
> >
> > hadoop jar /usr/lib/hbase/hbase.jar verifyrep --starttime=1397217601927
> > --endtime=1397228401927 --families=t 1 tablename 2>&1
> > 14/04/11 17:21:03 INFO mapred.JobClient:     BADROWS=178
> >
> > After that I run the command with the same start time and the actual
> > timestamp an end time, so the time frame is greater
> > but with the same start time. And now I got :
> >
> > hadoop jar /usr/lib/hbase/hbase.jar verifyrep --starttime=1397217601927
> > --endtime=1397230074876 --families=t 1 tablename 2>&1
> > 14/04/11 17:28:28 INFO mapred.JobClient:     GOODROWS=184
> >
> > Is there something wrong with the command?
> > In our metrics i could not see that three is an Issue at that time.
> >
> > We are a little bit confused about the endtime. In all documents they talk
> > about stoptime.
> > But we found that in the job configuration there is no parameter called
> > stoptime.
> > We found the "verifyrep.startTime" which hold the value of the starttime
> > in our command and
> > "verifyrep.endTime" which is alway 0 when we use stoptime in the command.
> > So we decided to use endtime
> >
> > Even in the code
> > http://hbase.apache.org/xref/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.html
> > they use: "static long endTime = Long.MAX_VALUE;"
> >
> > Which name is the right on? endtime or stoptime?
> >
> > We use cdh 4.2.0.
> >
> > Regards Hansi
> >
> 

Mime
View raw message