hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: CleanerChore exception
Date Sun, 30 Dec 2012 22:42:53 GMT
I'm not sure I'm getting that.

It's recursive. So when you are on the parent directory, you don't
know yet if the child directory is empty or not. So you can't call the
delete() yet. If you call the delet() giving "true" for recurs, then
you might delete some files who just got created, which we want to
avoid.

IMHO.

2012/12/30, Ted <yuzhihong@gmail.com>:
> Thanks for your digging.
>
> Minor optimization would be to issue delete() on the parent directory so
> that there are fewer requests to namenode.
>
> Cheers
>
> On Dec 30, 2012, at 2:15 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org>
> wrote:
>
>> I did the change, pushed it and it cleaned my directories correctly.
>>
>> // if the directory doesn't exist or is empty, then we are done
>> if (children == null) return fs.delete(toCheck, false);
>>
>> The only thing is that I don't know what will fs.delete() return i
>> case the directory doesn't exist. But I think it's still correct to
>> return false if the directory doesn't exist because we can't really
>> delete something which doesn't exist...
>>
>> My opinion.
>>
>> So the patch is ready, easy one ;) Just waiting for Jesse's feedback
>> just in case.
>>
>> JM
>>
>> 2012/12/30, lars hofhansl <lhofhansl@yahoo.com>:
>>> Nothing has changed around this in 0.94.4 as far as I know.
>>>
>>>
>>>
>>>
>>> ________________________________
>>> From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
>>> To: user@hbase.apache.org
>>> Sent: Sunday, December 30, 2012 9:53 AM
>>> Subject: Re: CleanerChore exception
>>>
>>> I was going to move to 0.94.4 today ;) And yes I'm using 0.94.3. I
>>> might wait a bit in case some testing is required with my version.
>>>
>>> Is this what you are looking for? http://pastebin.com/N8Q0FMba
>>>
>>> I will keep the files for now since it seems it's not causing any
>>> major issue. That will allow some more testing if required.
>>>
>>> JM
>>>
>>>
>>> 2012/12/30, Ted Yu <yuzhihong@gmail.com>:
>>>> Looks like you're using 0.94.3
>>>>
>>>> The archiver is backport of:
>>>> HBASE-5547, Don't delete HFiles in backup mode
>>>>
>>>> Can you provide more the log where the IOE was reported using pastebin
>>>> ?
>>>>
>>>> Thanks
>>>>
>>>> On Sun, Dec 30, 2012 at 9:08 AM, Jean-Marc Spaggiari <
>>>> jean-marc@spaggiari.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have a "IOException" /hbase/.archive/table_name is non empty
>>>>> exception every minute on my logs.
>>>>>
>>>>> There is 30 directories under this directory. the main directory is
>>>>> from yesterday, but all sub directories are from December 10th, all
>>>>> the same time.
>>>>>
>>>>> What does this .archive directory is used for, and what should I do?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> JM
>>>>
>

Mime
View raw message