Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAPQV63WXALedTYNTomWZjsANUJejkFocf+DcxyHer8LDGuCy=w@mail.gmail.com>
References: 
 <CAPQV63Xo07HvGerJuFxT=84yhAppDyzBmPR_E-EASkDBwXhLkQ@mail.gmail.com>
	<-7459091258275308652@unknownmsgid>
	<CAPQV63WXALedTYNTomWZjsANUJejkFocf+DcxyHer8LDGuCy=w@mail.gmail.com>
Date: Mon, 17 Dec 2012 07:15:32 -0500
Message-ID: 
 <CAPQV63VUWV0XrhLUXiYvdNm1rFRF+7Rkm_OMQEpcyiQUzD1Pvg@mail.gmail.com>
Subject: Re: MR missing lines
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
To: user@hbase.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

The job run the morning, and of course, this time, all the rows got process=
ed ;)

So I will give it few other tries and will keep you posted if I'm able
to reproduce that again.

Thanks,

JM

2012/12/16, Jean-Marc Spaggiari <jean-marc@spaggiari.org>:
> Thanks for the suggestions.
>
> I already have logs to display all the exepctions and there is
> nothing. I can't display the work done, there is to much :(
>
> I have counters "counting" the rows processed and they match what is
> done, minus what is not processed. I have just added few other
> counters. One right at the beginning, and one to count what are the
> records remaining on the delete list, as suggested.
>
> I will run the job again tomorrow, see the result and keep you posted.
>
> JM
>
>
> 2012/12/16, Asaf Mesika <asaf.mesika@gmail.com>:
>> Did you check the returned array of the delete method to make sure all
>> records sent for delete have been deleted?
>>
>> Sent from my iPhone
>>
>> On 16 =D7=91=D7=93=D7=A6=D7=9E 2012, at 14:52, Jean-Marc Spaggiari <jean=
-marc@spaggiari.org>
>> wrote:
>>
>>> Hi,
>>>
>>> I have a table where I'm running MR each time is exceding 100 000 rows.
>>>
>>> When the target is reached, all the feeding process are stopped.
>>>
>>> Yesterday it reached 123608 rows. So I stopped the feeding process,
>>> and ran the MR.
>>>
>>> For each line, the MR is creating a delete. The delete is placed on a
>>> list, and when the list reached 10 elements, it's sent to the table.
>>> In the clean method, the list is sent to the table if there is any
>>> element in it.
>>>
>>> So at the en of the MR, I should have an empty table.
>>>
>>> The table is splitted over 128 regions. And I have 8 region servers.
>>>
>>> What is disturbing me is that after the MR, I had 38 lines remaining
>>> on the table. the MR took 348 minutes to run. So I ran the MR again,
>>> which this time took 2 minutes, and now I have 1 row remaining in the
>>> table.
>>>
>>> I looked at the logs (for the 38 lines run) and there is nothing in
>>> it. There is some scanner timeout exception for the run of the 100K
>>> rows.
>>>
>>> I'm running HBase 0.94.3.
>>>
>>> I will hava another 100K rows today, so I will re-run the job. I will
>>> increase the timeout to make sure I got no exception, but even when I
>>> ran the 38 lines with no exception one was remaining...
>>>
>>> Any idea why and where I can seach? It's not really an issue for me
>>> since I can just re-run the job, but this might be an issue for some
>>> others.
>>>
>>> JM
>>
>