Return-Path: Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: (qmail 57069 invoked from network); 9 Nov 2010 16:28:30 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 9 Nov 2010 16:28:30 -0000 Received: (qmail 46317 invoked by uid 500); 9 Nov 2010 16:29:00 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 46158 invoked by uid 500); 9 Nov 2010 16:28:59 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 46150 invoked by uid 99); 9 Nov 2010 16:28:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Nov 2010 16:28:59 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of wlangiewicz@gmail.com designates 209.85.161.48 as permitted sender) Received: from [209.85.161.48] (HELO mail-fx0-f48.google.com) (209.85.161.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Nov 2010 16:28:52 +0000 Received: by fxm7 with SMTP id 7so5196399fxm.35 for ; Tue, 09 Nov 2010 08:28:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=3wvCa421ZThAd++csvNm4XLN+Bnc7ynPJsW4DaYloOI=; b=fB4zVGftLDFLim481Z0SQoNg+cnKDmSz8PYg14eU566LhPylOmPUgOoWFy8MVd7nCY 65Pq9Lgr8nrVOlDL5k9pBCBtJynDYQCPA6+XDhi1MTTZCw4Sz+I0BSjv+TKi0jk9Wvuz DtDr99eN+bXrI3KOlzOl7RYiX6g1BSprRjX9w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; b=FlKMXwbNIX2S7kgCL8zIfbR3npOqBRJNJGkDnHnPm6H1yuypXXORVjrLfndvo3JCZF azEe3oTc5vJeZI3LWw062baTJqVc6JiJcq31TkZj903v4Ffp8PIfhN2d7JndoA6nwiFV Xj2qi9/rYJ4h/1JSORlowU8v2aHBsBX2eSdu4= Received: by 10.223.83.133 with SMTP id f5mr5336156fal.29.1289320110659; Tue, 09 Nov 2010 08:28:30 -0800 (PST) Received: from [172.19.30.231] (static.nk-net.pl [195.88.186.3]) by mx.google.com with ESMTPS id 15sm776230fal.22.2010.11.09.08.28.29 (version=SSLv3 cipher=RC4-MD5); Tue, 09 Nov 2010 08:28:29 -0800 (PST) Message-ID: <4CD976AC.6030203@gmail.com> Date: Tue, 09 Nov 2010 17:28:28 +0100 From: Wojciech Langiewicz User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: general@hadoop.apache.org Subject: Re: cleaning up 'hadoop.tmp.dir' ? References: <4CD940FC.5000306@gmail.com> <4CD954D1.1080403@gmail.com> <4CD95AC5.2010609@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit W dniu 09.11.2010 17:23, Aaron Myers pisze: > On Tue, Nov 9, 2010 at 6:29 AM, Wojciech Langiewicz > wrote: > >> >> What might be causing situation where I have about 5TB in HDFS and hadoop >> tmp dirs have about 16TB in total? >> > > If indeed this is the block data of your HDFS files, then this makes perfect > sense. HDFS by default replicates every block 3 times, so ~5TB used in HDFS > is ~15TB raw on disk. You are right, I wonder why didn't I though about it before. Thanks for all the answers:) But name of this option 'hadoop.tmp.dir' is at least a little confusing. -- Wojciech Langiewicz