hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Regarding Hadoop Log Retention for Hadoop 0.20.2
Date Wed, 07 Dec 2011 11:38:06 GMT
I'm using CDH3 here and I do not see that option in my
log4j.properties. Probably something remaining from an older
mechanism?

Good to know the actual property worked for you.

On Wed, Dec 7, 2011 at 4:18 PM, Sahana Bhat <sana.bhat@gmail.com> wrote:
> Hi Harsh,
>
>           Thanks for the reply. I realized that the property you mentioned
> was missing in my mapred-site.xml.
> I added the entry and it works just fine.
>
> Was my assumption that "hadoop.tasklog.logsRetainHours " in log4j.properties
> will do the same wrong? What is this property for in that case?
>
> Regards,
> Sahana
>
>
>
>
> On Wed, Dec 7, 2011 at 2:28 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>> Sahana,
>>
>> IIRC, "mapred.userlog.retain.hours" (24h default) controls this in my
>> environment and it seems to work fine on my cluster. Are you sure you
>> have tasklogs older than 24h lying around? It might even be a bug that
>> may have been fixed in the subsequent 0.20 releases that went out
>> recently.
>>
>> On Wed, Dec 7, 2011 at 1:53 PM, Sahana Bhat <sana.bhat@gmail.com> wrote:
>> > Hi,
>> >
>> >         We use Hadoop 0.20.2 version.The log4j.properties file has a
>> > property hadoop.tasklog.logsRetainHours (mentioned as 24 hours by
>> > default) which we have set to 12.Despite this property being set we
>> > still
>> > have old userlogs in the system.Any idea why the attempt directories
>> > (that
>> > have the syserr/sysout files) have not been cleaned up? Are we missing
>> > any
>> > other configuration?
>> >
>> > Any help on this topic would be great.
>> >
>> > Regards,
>> > Sahana
>>
>>
>>
>> --
>> Harsh J
>
>



-- 
Harsh J

Mime
View raw message