hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Hive scratch dir not cleaning up
Date Fri, 01 Jun 2012 13:53:23 GMT
The filecrush tool has a small utility called Clean that accepts and
age argument and deletes all the files in a directory older then a
certain time.

We use clean to clean up the tmp hdfs directories applications leave
remnants in.

Edward

On 6/1/12, Vinod Singh <vinod@vinodsingh.com> wrote:
> Yes, that is how I do. Though 1 month is too long, I keep it just 2 days.
>
> Thanks,
> Vinod
>
> http://blog.vinodsingh.com/
>
> On Fri, Jun 1, 2012 at 2:15 PM, Ruben de Vries
> <ruben.devries@hyves.nl>wrote:
>
>> So I should write a job which cleans up 1 month old results or something
>> like that?
>>
>> From: Vinod Singh [mailto:vinod@vinodsingh.com]
>> Sent: Friday, June 01, 2012 10:35 AM
>> To: user@hive.apache.org
>> Subject: Re: Hive scratch dir not cleaning up
>>
>> Hive deletes job contents from the scratch directory on completion of the
>> job. Though failed / killed jobs leave data there, which needs to be
>> removed manually.
>>
>> Thanks,
>> Vinod
>>
>> http://blog.vinodsingh.com/
>> On Fri, Jun 1, 2012 at 1:58 PM, Ruben de Vries <ruben.devries@hyves.nl>
>> wrote:
>> Hey Hivers,
>>
>> I’m almost ready to replace our old hadoop implementation with a
>> implementation using Hive,
>>
>> Now I’ve ran into (hopefully) my last problem; my /tmp/hive-hduser dir is
>> getting kinda big!
>> It doesn’t seem to cleanup this tmp files, googling for it I run into
>> some
>> tickets about a cleanup setting, should I enable this with the below
>> setting?
>> Why doesn’t it do that by default? Am I the only one somehow racking up a
>> lot of space with tmp files?
>>
>>
>>
>>
>> <property>
>>   <name>hive.start.cleanup.scratchdir</name>
>>   <value>true</value>
>> </property>
>>
>>
>

Mime
View raw message