Return-Path: 
 <user-return-13638-apmail-hadoop-common-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-common-user-archive@www.apache.org
Delivered-To: apmail-hadoop-common-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 606FC10B25
	for <apmail-hadoop-common-user-archive@www.apache.org>;
 Fri,  7 Feb 2014 08:44:55 +0000 (UTC)
Received: (qmail 47362 invoked by uid 500); 7 Feb 2014 08:44:46 -0000
Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org
Received: (qmail 46712 invoked by uid 500); 7 Feb 2014 08:44:36 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Delivered-To: moderator for user@hadoop.apache.org
Received: (qmail 43656 invoked by uid 99); 7 Feb 2014 08:41:35 -0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of alexndr.frolov@gmail.com
 designates 209.85.215.42 as permitted sender)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=0LnnMgF7cq4QY3VvJu80lKwdM+BWxW7FBdkU1pFnPp8=;
        b=We635mKy4Js2YjMLvspRl69D5JUAX/x8hsIJesSFWdxgRas+Jp89ocovkLkxsGYVSj
         gn0xfz14gThHT4B/gqr7YpYJfntvIbZjge5LkFxWBkCpwTwg0XStQ1dRrv/N2kVMH2dl
         Kf6O0PLBcddAUc+TY4lqZkunSKuTOcmXONml+Rwn3M/G8eXZWDP1//wiBy6AxVnM3Lul
         O1itKCLGJxw335k+V5S5K65nBFekSuJ3L9DrByr/ZrIRQ6xVnOTrwy0hAaxpgLnSf/Nn
         i+Vv/8xJWa3AQSA37+Y45uBEWp3TlXGhqoKn4Zfpryukqi3NU2kA+aErVc3zdeGEA+QP
         Xwtw==
MIME-Version: 1.0
X-Received: by 10.152.88.82 with SMTP id be18mr9018994lab.3.1391762467194;
 Fri, 07 Feb 2014 00:41:07 -0800 (PST)
In-Reply-To: 
 <CAAREzCmD-Y-adOX9i-9SMWzjTD8W7eoE07YXfMtgAoaYgBbnbQ@mail.gmail.com>
References: 
 <CAAREzCmD-Y-adOX9i-9SMWzjTD8W7eoE07YXfMtgAoaYgBbnbQ@mail.gmail.com>
Date: Fri, 7 Feb 2014 12:41:07 +0400
Message-ID: 
 <CAAREzCmwFz88JrNLTUBkSo9H2Rvtr3Eej89xy=kiLWez0B0hpw@mail.gmail.com>
Subject: Fwd: HDFS: file is not distributed after upload
From: Alexander Frolov <alexndr.frolov@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a11c36486780b3c04f1ccf38b
X-Virus-Checked: Checked by ClamAV on apache.org

--001a11c36486780b3c04f1ccf38b
Content-Type: text/plain; charset=ISO-8859-1

Hi, folks!

I've deployed hadoop (0.20.203.0rc1) on 8-node cluster. After uploading
file onto hdfs I've got this file only on one of the nodes instead of being
uniformly distributed across all nodes. What can be the issue?

$HADOOP_HOME/bin/hadoop dfs -copyFromLocal ../data/rmat-20.0
/user/frolo/input/rmat-20.0

$HADOOP_HOME/bin/hadoop dfs -stat "%b %o %r %n" /user/frolo/input/rmat-*
1220222968 67108864 1 rmat-20.0

$HADOOP_HOME/bin/hadoop dfsadmin -report
Configured Capacity: 2536563998720 (2.31 TB)
Present Capacity: 1642543419392 (1.49 TB)
DFS Remaining: 1641312030720 (1.49 TB)
DFS Used: 1231388672 (1.15 GB)
DFS Used%: 0.07%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Datanodes available: 8 (8 total, 0 dead)

Name: 10.10.1.15:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 131536928768 (122.5 GB)
DFS Remaining: 185533546496(172.79 GB)
DFS Used%: 0%
DFS Remaining%: 58.51%
Last contact: Fri Feb 07 12:10:27 MSK 2014


Name: 10.10.1.13:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 131533377536 (122.5 GB)
DFS Remaining: 185537097728(172.79 GB)
DFS Used%: 0%
DFS Remaining%: 58.52%
Last contact: Fri Feb 07 12:10:27 MSK 2014


Name: 10.10.1.17:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 120023924736 (111.78 GB)
DFS Remaining: 197046550528(183.51 GB)
DFS Used%: 0%
DFS Remaining%: 62.15%
Last contact: Fri Feb 07 12:10:27 MSK 2014


Name: 10.10.1.18:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 78510628864 (73.12 GB)
DFS Remaining: 238559846400(222.18 GB)
DFS Used%: 0%
DFS Remaining%: 75.24%
Last contact: Fri Feb 07 12:10:24 MSK 2014


Name: 10.10.1.14:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 131537530880 (122.5 GB)
DFS Remaining: 185532944384(172.79 GB)
DFS Used%: 0%
DFS Remaining%: 58.51%
Last contact: Fri Feb 07 12:10:27 MSK 2014


Name: 10.10.1.11:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 1231216640 (1.15 GB)
Non DFS Used: 84698116096 (78.88 GB)
DFS Remaining: 231141167104(215.27 GB)
DFS Used%: 0.39%
DFS Remaining%: 72.9%
Last contact: Fri Feb 07 12:10:24 MSK 2014


Name: 10.10.1.16:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 131537494016 (122.5 GB)
DFS Remaining: 185532981248(172.79 GB)
DFS Used%: 0%
DFS Remaining%: 58.51%
Last contact: Fri Feb 07 12:10:27 MSK 2014


Name: 10.10.1.12:50010
Decommission Status : Normal
Configured Capacity: 317070499840 (295.29 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 84642578432 (78.83 GB)
DFS Remaining: 232427896832(216.47 GB)
DFS Used%: 0%
DFS Remaining%: 73.3%
Last contact: Fri Feb 07 12:10:27 MSK 2014


Best,
  Alex

--001a11c36486780b3c04f1ccf38b
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_quote"><div dir=3D"ltr"><div>Hi, f=
olks!</div><div><br></div><div>I&#39;ve deployed hadoop (0.20.203.0rc1) on =
8-node cluster. After uploading file onto hdfs I&#39;ve got this file only =
on one of the nodes instead of being uniformly distributed across all nodes=
. What can be the issue?</div>

<div><br></div><div><div>$HADOOP_HOME/bin/hadoop dfs -copyFromLocal ../data=
/rmat-20.0 /user/frolo/input/rmat-20.0</div><div><br></div><div>$HADOOP_HOM=
E/bin/hadoop dfs -stat &quot;%b %o %r %n&quot; /user/frolo/input/rmat-*</di=
v>

<div>1220222968 67108864 1 rmat-20.0</div><div><br></div><div>$HADOOP_HOME/=
bin/hadoop dfsadmin -report=A0</div><div>Configured Capacity: 2536563998720=
 (2.31 TB)</div><div>Present Capacity: 1642543419392 (1.49 TB)</div><div>

DFS Remaining: 1641312030720 (1.49 TB)</div><div>DFS Used: 1231388672 (1.15=
 GB)</div><div>DFS Used%: 0.07%</div><div>Under replicated blocks: 0</div><=
div>Blocks with corrupt replicas: 0</div><div>Missing blocks: 0</div><div>

<br></div><div>-------------------------------------------------</div><div>=
Datanodes available: 8 (8 total, 0 dead)</div><div><br></div><div>Name: <a =
href=3D"http://10.10.1.15:50010" target=3D"_blank">10.10.1.15:50010</a></di=
v>
<div>Decommission Status : Normal</div>
<div>Configured Capacity: 317070499840 (295.29 GB)</div><div>DFS Used: 2457=
6 (24 KB)</div><div>Non DFS Used: 131536928768 (122.5 GB)</div><div>DFS Rem=
aining: 185533546496(172.79 GB)</div><div>DFS Used%: 0%</div><div>DFS Remai=
ning%: 58.51%</div>

<div>Last contact: Fri Feb 07 12:10:27 MSK 2014</div><div><br></div><div><b=
r></div><div>Name: <a href=3D"http://10.10.1.13:50010" target=3D"_blank">10=
.10.1.13:50010</a></div><div>Decommission Status : Normal</div><div>Configu=
red Capacity: 317070499840 (295.29 GB)</div>

<div>DFS Used: 24576 (24 KB)</div><div>Non DFS Used: 131533377536 (122.5 GB=
)</div><div>DFS Remaining: 185537097728(172.79 GB)</div><div>DFS Used%: 0%<=
/div><div>DFS Remaining%: 58.52%</div><div>Last contact: Fri Feb 07 12:10:2=
7 MSK 2014</div>

<div><br></div><div><br></div><div>Name: <a href=3D"http://10.10.1.17:50010=
" target=3D"_blank">10.10.1.17:50010</a></div><div>Decommission Status : No=
rmal</div><div>Configured Capacity: 317070499840 (295.29 GB)</div><div>DFS =
Used: 24576 (24 KB)</div>

<div>Non DFS Used: 120023924736 (111.78 GB)</div><div>DFS Remaining: 197046=
550528(183.51 GB)</div><div>DFS Used%: 0%</div><div>DFS Remaining%: 62.15%<=
/div><div>Last contact: Fri Feb 07 12:10:27 MSK 2014</div><div><br></div>

<div><br></div><div>Name: <a href=3D"http://10.10.1.18:50010" target=3D"_bl=
ank">10.10.1.18:50010</a></div><div>Decommission Status : Normal</div><div>=
Configured Capacity: 317070499840 (295.29 GB)</div><div>DFS Used: 24576 (24=
 KB)</div>
<div>Non DFS Used: 78510628864 (73.12 GB)</div>
<div>DFS Remaining: 238559846400(222.18 GB)</div><div>DFS Used%: 0%</div><d=
iv>DFS Remaining%: 75.24%</div><div>Last contact: Fri Feb 07 12:10:24 MSK 2=
014</div><div><br></div><div><br></div><div>Name: <a href=3D"http://10.10.1=
.14:50010" target=3D"_blank">10.10.1.14:50010</a></div>

<div>Decommission Status : Normal</div><div>Configured Capacity: 3170704998=
40 (295.29 GB)</div><div>DFS Used: 24576 (24 KB)</div><div>Non DFS Used: 13=
1537530880 (122.5 GB)</div><div>DFS Remaining: 185532944384(172.79 GB)</div=
>

<div>DFS Used%: 0%</div><div>DFS Remaining%: 58.51%</div><div>Last contact:=
 Fri Feb 07 12:10:27 MSK 2014</div><div><br></div><div><br></div><div>Name:=
 <a href=3D"http://10.10.1.11:50010" target=3D"_blank">10.10.1.11:50010</a>=
</div>
<div>Decommission Status : Normal</div>
<div>Configured Capacity: 317070499840 (295.29 GB)</div><div>DFS Used: 1231=
216640 (1.15 GB)</div><div>Non DFS Used: 84698116096 (78.88 GB)</div><div>D=
FS Remaining: 231141167104(215.27 GB)</div><div>DFS Used%: 0.39%</div>
<div>
DFS Remaining%: 72.9%</div><div>Last contact: Fri Feb 07 12:10:24 MSK 2014<=
/div><div><br></div><div><br></div><div>Name: <a href=3D"http://10.10.1.16:=
50010" target=3D"_blank">10.10.1.16:50010</a></div><div>Decommission Status=
 : Normal</div>
<div>
Configured Capacity: 317070499840 (295.29 GB)</div><div>DFS Used: 24576 (24=
 KB)</div><div>Non DFS Used: 131537494016 (122.5 GB)</div><div>DFS Remainin=
g: 185532981248(172.79 GB)</div><div>DFS Used%: 0%</div><div>DFS Remaining%=
: 58.51%</div>

<div>Last contact: Fri Feb 07 12:10:27 MSK 2014</div><div><br></div><div><b=
r></div><div>Name: <a href=3D"http://10.10.1.12:50010" target=3D"_blank">10=
.10.1.12:50010</a></div><div>Decommission Status : Normal</div><div>Configu=
red Capacity: 317070499840 (295.29 GB)</div>

<div>DFS Used: 24576 (24 KB)</div><div>Non DFS Used: 84642578432 (78.83 GB)=
</div><div>DFS Remaining: 232427896832(216.47 GB)</div><div>DFS Used%: 0%</=
div><div>DFS Remaining%: 73.3%</div><div>Last contact: Fri Feb 07 12:10:27 =
MSK 2014</div>

</div><div><br></div><div><br></div><div>Best,</div><div>=A0 Alex</div></di=
v>
</div><br></div>

--001a11c36486780b3c04f1ccf38b--