hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kartashov, Andy" <Andy.Kartas...@mpac.ca>
Subject RE: Log files occupy lot of Disk size
Date Fri, 23 Nov 2012 21:44:34 GMT
Every so often I simply wipe out the whole directory content:


I also clean /home/mapred/local/* and /tmp

-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Friday, November 23, 2012 2:43 AM
To: <user@hadoop.apache.org>
Subject: Re: Log files occupy lot of Disk size

Lower your log levels if you do not need all that verbosity. You can control log retention,
max sizes to keep, max number of files to keep, and logging levels, etc. via each components'
log4j.properties file.

On Fri, Nov 23, 2012 at 12:42 PM, iwannaplay games <funnlearnforkids@gmail.com> wrote:
> If i delete the log file without stopping the cluster  won't it
> terminate the session.
> On 11/23/12, Mohammad Tariq <dontariq@gmail.com> wrote:
>> Hi there,
>>     You can write a small job or some script which periodically
>> checks for the log growth and performs the delete after certain threshold.
>> Regards,
>>     Mohammad Tariq
>> On Fri, Nov 23, 2012 at 12:28 PM, iwannaplay games <
>> funnlearnforkids@gmail.com> wrote:
>>> Hi,
>>> Everytime i query hbase or hive ,there is a significant growth in my
>>> log files and it consumes lot of space from my hard disk....(Approx
>>> 40
>>> gb)
>>> So i stop the cluster ,delete all the logs and free the space and
>>> then again start the cluster to start my work.
>>> Is there any other solution coz i cannot restart the cluster everyday.

Harsh J
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and
may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not
the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe
qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite.
Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
View raw message