Return-Path: Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: (qmail 19539 invoked from network); 31 Aug 2009 16:48:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 31 Aug 2009 16:48:24 -0000 Received: (qmail 87593 invoked by uid 500); 31 Aug 2009 16:48:21 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 87520 invoked by uid 500); 31 Aug 2009 16:48:21 -0000 Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-user@hadoop.apache.org Delivered-To: mailing list common-user@hadoop.apache.org Received: (qmail 87510 invoked by uid 99); 31 Aug 2009 16:48:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Aug 2009 16:48:21 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jim.twensky@gmail.com designates 209.85.146.182 as permitted sender) Received: from [209.85.146.182] (HELO wa-out-1112.google.com) (209.85.146.182) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Aug 2009 16:48:12 +0000 Received: by wa-out-1112.google.com with SMTP id j32so543182waf.29 for ; Mon, 31 Aug 2009 09:47:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=RELWJCqXKp8LomxkK1npAYBYpIuE29QTwNHWc0GciqQ=; b=I8CttBlP+qIqyOcv1uk32d88ozHDMHkOYUh9Ebay9mAGVteNqM2xaIkZvG94XBdiCU TKdMsJqJVA/EQlxvfKZHuUIbocUn3xyquEYBQHEqXOuW/Nb95s5lF7H9yR9n7yl18kXW WFFPezerPiKdSDcbd++Ga/bRMg4RXGetFAtQU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=sDVu7JtT8S03t9d2K7Hpwp0ELRIESkF/3ChunEr3TYptn/ObH6Z9AIkA/BKyP9EePe 7eUpDkqtzmQxj1tcMGI/rXNbcx6HHVpG/LFxs9ckwnYaLfgvQkKvr99PImONKoOR1vFd l7ccv2MbuvQqsshZWIpRN1jXNTqHpyEsu/PUI= MIME-Version: 1.0 Received: by 10.114.31.14 with SMTP id e14mr523223wae.78.1251737270845; Mon, 31 Aug 2009 09:47:50 -0700 (PDT) In-Reply-To: <4061df20908310219w17e32c36mc57eabd31b86af9c@mail.gmail.com> References: <8211a1320908241749x71f4d762j3256e6a0be0e22f3@mail.gmail.com> <4061df20908310219w17e32c36mc57eabd31b86af9c@mail.gmail.com> Date: Mon, 31 Aug 2009 09:47:50 -0700 Message-ID: <7a8854060908310947n5784768byc6e00908882d33@mail.gmail.com> Subject: Re: Does hadoop delete the intermediate data From: Jim Twensky To: common-user@hadoop.apache.org Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Hi Jeff, The problem may also be related to the large log files if you use the cluster for too many jobs. Check out your hadoop log directory and see how big it is. You can decrease the maximum size of a log file using one of the hadoop configuration files under conf. Jim On Mon, Aug 31, 2009 at 2:19 AM, Chandraprakash Bhagtani wrote: > Hadoop does delete the intermediate data after the job completes. > Jobtracker sends signal to Tasktracker to delete intermediate data > when the job completes. > > The problem in your case might be some of your running job might not > have been killed gracefully or Jobtracker failed for some reason. > > -- > Thanks & Regards, > Chandra Prakash Bhagtani, > > On Tue, Aug 25, 2009 at 6:19 AM, zhang jianfeng wrote: > >> Hi all, >> >> I found my cluster=92s space usage increase over time although I did not >> upload new data. =A0And there's a lot of files under folder /tmp . >> >> So I guess hadoop won=92t delete the intermediate data(output of mapper)= . >> >> Am I right ? >> >> >> Thank you. >> >> Jeff zhang >> >