Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 69F346AFA for ; Thu, 9 Jun 2011 17:21:25 +0000 (UTC) Received: (qmail 66094 invoked by uid 500); 9 Jun 2011 17:21:24 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 66019 invoked by uid 500); 9 Jun 2011 17:21:24 -0000 Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-user@hadoop.apache.org Delivered-To: mailing list hdfs-user@hadoop.apache.org Received: (qmail 66011 invoked by uid 99); 9 Jun 2011 17:21:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jun 2011 17:21:24 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=HTML_MESSAGE,NO_RDNS_DOTCOM_HELO,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [216.145.54.172] (HELO mrout2.yahoo.com) (216.145.54.172) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jun 2011 17:21:15 +0000 Received: from SP2-EX07CAS03.ds.corp.yahoo.com (sp2-ex07cas03.corp.sp2.yahoo.com [98.137.59.35]) by mrout2.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id p59HKRc0039814 for ; Thu, 9 Jun 2011 10:20:27 -0700 (PDT) Received: from SP2-EX07VS04.ds.corp.yahoo.com ([98.137.59.33]) by SP2-EX07CAS03.ds.corp.yahoo.com ([98.137.59.35]) with mapi; Thu, 9 Jun 2011 10:20:27 -0700 From: Koji Noguchi To: "hdfs-user@hadoop.apache.org" Date: Thu, 9 Jun 2011 10:20:24 -0700 Subject: Re: When rmr and rm strike Thread-Topic: When rmr and rm strike Thread-Index: AcwmtPObsbPE5ae5QauM3AYv33maogAFJDju Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-Entourage/13.9.0.110114 acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CA164EE829F65knoguchiyahooinccom_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_CA164EE829F65knoguchiyahooinccom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable > What i have found is that the setting needs to be on the client side conf= iguration and not necessarily required on the namenode. > As long as the hadoop configuration on the node from which you run the -r= mr has this setting, the files will be moved into the .Trash directory. > Can anyone else confirm this behaviour? > Move to Trash is a complete client-side behavior. So, config on the client = side is sufficient. However, purging the Trash periodically is done by the namenode. So unless= namenode has the same config setup, those Trash remains forever. Koji On 6/9/11 7:52 AM, "sridhar basam" wrote: On Thu, Jun 9, 2011 at 10:42 AM, Florin P wrote: Hello! Your answers solved my problem. Below are the steps: 1. In the core-site.xml of the job-tracker add the following entry: fs.trash.interval 1440 2. Shutdown job-tracker (job tracker machine hadoop/bin/stop-mapred.sh) 3. Shutdown namenode (namenode machine hadoop/bin/stop-dfs.sh) 4. Start namenode (namenode machine hadoop/bin/start-dfs.sh) 5. Start jobtracker (job tracker machine hadoop/bin/start-mapred.sh) What i have found is that the setting needs to be on the client side config= uration and not necessarily required on the namenode. As long as the hadoo= p configuration on the node from which you run the -rmr has this setting, t= he files will be moved into the .Trash directory. Can anyone else confirm t= his behaviour? Observation. I've added the property in the core-site.xml of the namenode, = but no effect. It worked only when I've added in the core-site.xml of the j= obtracker. Were you by chance doing the -rmr on the jobtracker? Sridhar --_000_CA164EE829F65knoguchiyahooinccom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: When rmr and rm strike > What i have found is that the setting needs to be on the client = side configuration and not necessarily required on the namenode. =A0
> As long as the hadoop configuration on the node from which you run the= -rmr has this setting, the files will be moved into the .Trash directory. =
> Can anyone else confirm this behaviour?
>
Move to Trash is a complete client-side behavior. So, config on the client = side is sufficient.
However, purging the Trash periodically is done by the namenode.  So u= nless namenode has the same config setup, those Trash remains forever.

Koji


On 6/9/11 7:52 AM, "sridhar basam" <= sri@basam.org> wrote:



On Thu, Jun 9, 2011 at 10:42 AM, Florin P <florinpico@yahoo.com> wrote:
Hello!
 =A0 Your answers solved my problem.
Below are the steps:
1. In the core-site.xml of the job-tracker add the following =A0entry:

<property>
 =A0 =A0<name>fs.trash.interval</name>
 =A0 =A0<value>1440</value>
</property>

2. Shutdown job-tracker (job tracker machine hadoop/bin/stop-mapred.sh)

3. Shutdown namenode (namenode machine hadoop/bin/stop-dfs.sh)

4. Start namenode (namenode machine hadoop/bin/start-dfs.sh)

5. Start jobtracker (job tracker machine hadoop/bin/start-mapred.sh)

Observation. I've added the property in the core-site.xml of the namenode, = but no effect. It worked only when I've added in the core-site.xml of the j= obtracker.