Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of vinodkv@hortonworks.com
 designates 64.78.52.187 as permitted sender)
Subject: Re: Retention policy on Yarn application's file cache
MIME-Version: 1.0
From: Vinod Kumar Vavilapalli <vinodkv@hortonworks.com>
To: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Thread-Topic: Retention policy on Yarn application's file cache
Thread-Index: AQHQaAw/cXF16+hEZESUgzL7J68Ijp0v1YOA
Date: Thu, 26 Mar 2015 22:56:46 +0000
Message-ID: <99373028-7670-4283-860A-6A92689D5195@hortonworks.com>
References: <D139C8CA.3408A%nramesh@linkedin.com>
In-Reply-To: <D139C8CA.3408A%nramesh@linkedin.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="Windows-1252"
Content-ID: <E165BBF86284E244B6173B7215428728@exch080.serverpod.net>
Content-Transfer-Encoding: quoted-printable

Log directories are treated separate from the local directories.

Retention of files in the local cache is debug-delay-sec which is an admin =
server-side configuration. Applications cannot control it.

+Vinod

On Mar 26, 2015, at 2:31 PM, Navina Ramesh <nramesh@linkedin.com.INVALID> w=
rote:

> Hi yarn-dev,
> I am trying to understand the extent to which the NMs support retaining a=
pplication related cache.
>=20
> =93yarn.nodemanager.delete.debug-delay-sec=94 can be used to control the =
number of seconds after an application finishes before the NM's DeletionSer=
vice will delete the application's localized file directory and log directo=
ry.
>=20
> Is there a way to control the retention period for the localized file dir=
ectory independent of the log directory ? That is, if I want to retain cert=
ain application data longer than the log's retention, is there a way to do =
it in YARN ?
>=20
> Thanks!
> Navina