Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 03D7FDAB9 for ; Wed, 28 Nov 2012 14:47:52 +0000 (UTC) Received: (qmail 70677 invoked by uid 500); 28 Nov 2012 14:47:47 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 70567 invoked by uid 500); 28 Nov 2012 14:47:46 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 70539 invoked by uid 99); 28 Nov 2012 14:47:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Nov 2012 14:47:46 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of balijamahesh.mca@gmail.com designates 209.85.216.41 as permitted sender) Received: from [209.85.216.41] (HELO mail-qa0-f41.google.com) (209.85.216.41) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Nov 2012 14:47:39 +0000 Received: by mail-qa0-f41.google.com with SMTP id c26so5380095qad.14 for ; Wed, 28 Nov 2012 06:47:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=jsh17cm6J3fiZzKTPrd3YIOZKB996F8+lj8I12psWoQ=; b=NX2+g0RVDjnQs6q2bQYbIoS73cKplMgtvk85jc7dKSyEuzI96hD8HzdffdwYaG6mEu LjmuqsJ9zv+GGspXDRMc5M1FMvXmhdRWYE/eoUyttbG4c54hJyOmWOdTaIG+JCGn/ZXq oWtsoGA0lkFjLzzzoUG7GXtIsIau8rVDMZeEn6+r/v7sibGfg0Kf3IAd+6Zo4RrkEN1h JzXh7dr/k6xkw/J74/cXgIYNIa9yELMiqSpk/9e3y0igFipsq0qanRePCbZ6OHqrUpdR PGLP+pdGMH/BSOEXANgeP9XmBa/o6tU23mzcUd/hWq/lNHMaZBDdcQDjdTZAIONctjTu F5zQ== MIME-Version: 1.0 Received: by 10.49.4.193 with SMTP id m1mr23584892qem.38.1354114038479; Wed, 28 Nov 2012 06:47:18 -0800 (PST) Received: by 10.49.60.69 with HTTP; Wed, 28 Nov 2012 06:47:18 -0800 (PST) In-Reply-To: <20121128141352.201930@gmx.net> References: <20121128141352.201930@gmx.net> Date: Wed, 28 Nov 2012 20:17:18 +0530 Message-ID: Subject: Re: discrepancy du in dfs are fs From: Mahesh Balija To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7bb043ac3fb9f404cf8f3e5a X-Virus-Checked: Checked by ClamAV on apache.org --047d7bb043ac3fb9f404cf8f3e5a Content-Type: text/plain; charset=ISO-8859-1 Hi Chris, Can you try the following in your local machine, du -b myfile.txt and compare this with the hadoop fs -du myfile.txt. Best, Mahesh Balija, Calsoft Labs. On Wed, Nov 28, 2012 at 7:43 PM, wrote: > > Hi all, > > I wonder wy there is a difference between "du" on HDFS and "get" + "du" on > my local machnine. > > Here is an example: > > hadoop fs -du myfile.txt > > 81355258 > > hadoop fs -get myfile.txt . > du myfile.txt > > 34919 > > --- nevertheless --- > > hadoop fs -cat myfile.txt | wc -l > > 4789943 > > cat myfile.txt | wc -l > > 4789943 > > > Any idea? > Thanks. > Chris > --047d7bb043ac3fb9f404cf8f3e5a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Chris,

=A0=A0=A0=A0=A0=A0=A0=A0=A0 Can you try the following in y= our local machine,

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 du -b = myfile.txt

=A0=A0=A0=A0=A0=A0=A0=A0=A0 and compare this with the ha= doop fs -du myfile.txt.

Best,
Mahesh Balija,
Calsoft Labs.

On Wed, Nov 28, 2012 at 7:43 PM, <list= enbruder@gmx.net> wrote:

Hi all,

I wonder wy there is a difference between "du" on HDFS and "= get" + "du" on my local machnine.

Here is an example:

hadoop fs -du myfile.txt
> 81355258

hadoop fs -get myfile.txt .
du myfile.txt
> 34919

--- nevertheless ---

hadoop fs -cat =A0myfile.txt | wc -l
> 4789943

cat myfile.txt | wc -l
> 4789943


Any idea?
Thanks.
Chris

--047d7bb043ac3fb9f404cf8f3e5a--