Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of vinodkv@hortonworks.com
 designates 209.85.220.46 as permitted sender)
From: Vinod Kumar Vavilapalli <vinodkv@hortonworks.com>
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_4E36DD4D-5907-4E29-BA08-7E215665717D"
Subject: Re: Auto clean DistCache?
Date: Tue, 26 Mar 2013 13:43:38 -0700
In-Reply-To: 
 <CAPQV63UV80T54mzwWbeyKkZET7gmpcLrS-heGHsdP0-TRRQ2Ng@mail.gmail.com>
To: user@hadoop.apache.org
References: 
 <CAPQV63UV80T54mzwWbeyKkZET7gmpcLrS-heGHsdP0-TRRQ2Ng@mail.gmail.com>
Message-Id: <39971159-ACE1-4D74-96E0-21B5B0B56E48@apache.org>


--Apple-Mail=_4E36DD4D-5907-4E29-BA08-7E215665717D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


You can control the limit of these cache files, the default is 10GB =
(value of 10737418240L): Try changing local.cache.size or =
mapreduce.tasktracker.cache.local.size in mapred-site.xml

Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Mar 25, 2013, at 5:16 PM, Jean-Marc Spaggiari wrote:

> Hi,
>=20
> Each time my MR job is run, a directory is created on the TaskTracker
> under mapred/local/taskTracker/hadoop/distcache (based on my
> configuration).
>=20
> I looked at the directory today, and it's hosting thousands of
> directories and more than 8GB of data there.
>=20
> Is there a way to automatically delete this directory when the job is =
done?
>=20
> Thanks,
>=20
> JM


--Apple-Mail=_4E36DD4D-5907-4E29-BA08-7E215665717D
Content-Transfer-Encoding: 7bit
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><br></div><div>You can control the limit of these cache files, the default is 10GB (value of 10737418240L): Try changing local.cache.size or&nbsp;mapreduce.tasktracker.cache.local.size in mapred-site.xml</div><br><div apple-content-edited="true">
<div><div class="">Thanks,</div><div class="">+Vinod Kumar Vavilapalli</div><div class="">Hortonworks Inc.<br class=""><a href="http://hortonworks.com/" class="">http://hortonworks.com/</a></div></div>
</div>

<br><div><div>On Mar 25, 2013, at 5:16 PM, Jean-Marc Spaggiari wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi,<br><br>Each time my MR job is run, a directory is created on the TaskTracker<br>under mapred/local/taskTracker/hadoop/distcache (based on my<br>configuration).<br><br>I looked at the directory today, and it's hosting thousands of<br>directories and more than 8GB of data there.<br><br>Is there a way to automatically delete this directory when the job is done?<br><br>Thanks,<br><br>JM<br></div></blockquote></div><br></body></html>
--Apple-Mail=_4E36DD4D-5907-4E29-BA08-7E215665717D--