Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A969010EA7 for ; Fri, 24 Jan 2014 22:23:41 +0000 (UTC) Received: (qmail 23429 invoked by uid 500); 24 Jan 2014 22:23:33 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 23262 invoked by uid 500); 24 Jan 2014 22:23:32 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 23254 invoked by uid 99); 24 Jan 2014 22:23:32 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Jan 2014 22:23:32 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [74.125.82.180] (HELO mail-we0-f180.google.com) (74.125.82.180) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Jan 2014 22:23:26 +0000 Received: by mail-we0-f180.google.com with SMTP id q59so3304976wes.11 for ; Fri, 24 Jan 2014 14:23:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=FrDgwr6JyGlOUTsJK4QTV5xrCW5UYqgj26fhNbuBjuU=; b=k0WYnPuB3hh3qXWsRqlvYNCoBpDqrurMrAT8uzO68jpec+cFsd71jTZPQDzXrwi55L poVho4+Ht7yMVpchkjg+gMOyMID2MOhoElZ1Y0gsFhbOw79w1Ol6F3KuCt6fBp/EYvDR Sg9NGmIQvGFj2WbKHYCdR7ZrgIRTP5Ql1rDKlyLVvuK1pzXL7xscWmCUIeiji1xwclt2 ip0ZTihXWlxRdBHlG5pnZdJRsCNPZ5DdaUxxvZGZdWHYhW6fT8zX7cOBzUE1gF/YmPnO YpgewBesN7oMey6jaXI5WthIs0Elw4Ti6+WFRMAaPglPN7N9L9KtcJ62BVuyrB1EkRW7 OH4Q== X-Gm-Message-State: ALoCoQmx5KJ3x5ehHWv8bZBUb/6AP4s/4sQ/z40v6mnHeZMMLpAZOH4/AceLYwDMAe44cKWcTFP0 MIME-Version: 1.0 X-Received: by 10.180.97.10 with SMTP id dw10mr4672068wib.25.1390602185735; Fri, 24 Jan 2014 14:23:05 -0800 (PST) Received: by 10.194.8.71 with HTTP; Fri, 24 Jan 2014 14:23:05 -0800 (PST) X-Originating-IP: [50.162.32.156] In-Reply-To: <13EE4E91-98E0-44E6-8D2C-AB815E6CDBA3@netapp.com> References: <9C13282BFA8E014D9093B25CA431321C1511A2AA@SACEXCMBX06-PRD.hq.netapp.com> <9C13282BFA8E014D9093B25CA431321C1511A3DF@SACEXCMBX06-PRD.hq.netapp.com> <13EE4E91-98E0-44E6-8D2C-AB815E6CDBA3@netapp.com> Date: Fri, 24 Jan 2014 16:23:05 -0600 Message-ID: Subject: Re: hdfs fsck -locations From: Mark Kerzner To: Hadoop User Content-Type: multipart/alternative; boundary=f46d0443064e4e158304f0becd49 X-Virus-Checked: Checked by ClamAV on apache.org --f46d0443064e4e158304f0becd49 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Yes, Rodrigo, that's what I was looking for. So in my install I somehow don't have it at all. Was asked by my students, so I got the answer. Mark On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo < Rodrigo.Nascimento@netapp.com> wrote: > Mark, > > there we go ;-) > > Rodrigo Nascimento > Systems Engineer @ Brazil > Mobile +55 11 991.873.810 > > Sent from my iPhone > > Begin forwarded message: > > *From:* "Nascimento, Rodrigo" > *Date:* 24 de janeiro de 2014 15:59:33 BRST > *To:* "user@hadoop.apache.org" > *Subject:* *RE: hdfs fsck -locations* > > Hi Mark, > > It is a sample from my sandbox. Your question is about the part that is i= n > RED at the output below, right? > > [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations > Connecting to namenode via http://sandbox.hortonworks.com:50070 > FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path > /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014 > . > */user/ambari-qa/passwd: Under replicated > BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replica= s > is 3 but found 1 replica(s).* > Status: HEALTHY > Total size: 1708 B > Total dirs: 0 > Total files: 1 > Total symlinks: 0 > Total blocks (validated): 1 (avg. block size 1708 B) > Minimally replicated blocks: 1 (100.0 %) > Over-replicated blocks: 0 (0.0 %) > Under-replicated blocks: 1 (100.0 %) > Mis-replicated blocks: 0 (0.0 %) > Default replication factor: 3 > Average block replication: 1.0 > Corrupt blocks: 0 > Missing replicas: 2 (66.666664 %) > Number of data-nodes: 1 > Number of racks: 1 > FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds > > > The filesystem under path '/user/ambari-qa/passwd' is HEALTHY > [root@sandbox ~]# > > Rod Nascimento > > ------------------------------ > *From:* Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com] > *Sent:* Friday, January 24, 2014 3:34 PM > *To:* user@hadoop.apache.org > *Subject:* RE: hdfs fsck -locations > > I=92m not seeing locations flag yet. > > > > *Rod Nascimento* > > *Systems Engineer @ Brazil* > > > > *People **don=92t** buy **WHAT** you do. They buy **WHY** you do it.* > > > > *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com] > > *Sent:* Friday, January 24, 2014 3:16 PM > *To:* Hadoop User > *Subject:* Re: hdfs fsck -locations > > > > Sorry, did not copy the full command > > > > hdfs fsck /user/mark/data/word_count.csv -locations > > Connecting to namenode via http://mark-7:50070 > > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 > > .Status: HEALTHY > > Total size: 7217 B > > Total dirs: 0 > > Total files: 1 > > Total blocks (validated): 1 (avg. block size 7217 B) > > Minimally replicated blocks: 1 (100.0 %) > > Over-replicated blocks: 0 (0.0 %) > > Under-replicated blocks: 0 (0.0 %) > > Mis-replicated blocks: 0 (0.0 %) > > Default replication factor: 1 > > Average block replication: 1.0 > > Corrupt blocks: 0 > > Missing replicas: 0 (0.0 %) > > Number of data-nodes: 1 > > Number of racks: 1 > > FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds > > > > > > The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY > > > > > > On Fri, Jan 24, 2014 at 11:08 AM, Harsh J wrote: > > Sorry, but what was the question? I also do not see a locations option > flag. > > On Jan 24, 2014 7:17 PM, "Mark Kerzner" wrote: > > Here is an example > > > > hdfs fsck /user/mark/data/word_count.csv > > Connecting to namenode via http://mark-7:50070 > > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 > > .Status: HEALTHY > > Total size: 7217 B > > Total dirs: 0 > > Total files: 1 > > Total blocks (validated): 1 (avg. block size 7217 B) > > Minimally replicated blocks: 1 (100.0 %) > > Over-replicated blocks: 0 (0.0 %) > > Under-replicated blocks: 0 (0.0 %) > > Mis-replicated blocks: 0 (0.0 %) > > Default replication factor: 1 > > Average block replication: 1.0 > > Corrupt blocks: 0 > > Missing replicas: 0 (0.0 %) > > Number of data-nodes: 1 > > Number of racks: 1 > > FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds > > > > > > On Fri, Jan 24, 2014 at 4:34 AM, Harsh J wrote: > > Hi Mark, > > Yes, the locations are shown as IP. > > > On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner > wrote: > > Hi, > > > > hdfs fsck -locations > > > > is supposed to show every block with its location? Is location the ip o= f > the > > datanode? > > > > Thank you, > > Mark > > > -- > Harsh J > > > > > > --f46d0443064e4e158304f0becd49 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Yes, Rodrigo,

that's what I was loo= king for. So in my install I somehow don't have it at all. Was asked by= my students, so I got the answer.

Mark


On Fri, Jan 2= 4, 2014 at 4:00 PM, Nascimento, Rodrigo <Rodrigo.Nascimento@ne= tapp.com> wrote:
Mark,

there we go ;-)

Rodrigo Nascimento
Systems Engineer @ Brazil

Sent from my iPhone

Begin forwarded message:

From: "Nascimento, Rodrigo" <Rodrigo.Nascimento@netapp.com= >
Date: 24 de janeiro de 2014 15:59:33 BRST
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Subject: RE: hdfs fsck -locations

Hi Mark,

It is a sample from my sandbox. Your question is about the part that is in = RED at the output below, right?

[root@sandbox ~]# hdfs fsck /user/ambari-qa/pass= wd=A0 -locations
Connecting to namenode via http://sandbox.hortonworks.com:50070
FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan= 24 09:53:43 PST 2014
.
/user/ambari-qa/passwd:=A0 Under replicated BP-15= 78958328-10.0.2.15-1382306880516:blk_10737424= 64_1640. Target Replicas is 3 but found 1 replica(s).
Status: HEALTHY
=A0Total size:=A0=A0=A0 1708 B
=A0Total dirs:=A0=A0=A0 0
=A0Total files:=A0=A0=A0 1
=A0Total symlinks:=A0=A0=A0 =A0=A0=A0 0
=A0Total blocks (validated):=A0=A0=A0 1 (avg. block size 1708 B)
=A0Minimally replicated blocks:=A0=A0=A0 1 (100.0 %)
=A0Over-replicated blocks:=A0=A0=A0 0 (0.0 %)
=A0Under-replicated blocks:=A0=A0=A0 1 (100.0 %)
=A0Mis-replicated blocks:=A0=A0=A0 =A0=A0=A0 0 (0.0 %)
=A0Default replication factor:=A0=A0=A0 3
=A0Average block replication:=A0=A0=A0 1.0
=A0Corrupt blocks:=A0=A0=A0 =A0=A0=A0 0
=A0Missing replicas:=A0=A0=A0 =A0=A0=A0 2 (66.666664 %)
=A0Number of data-nodes:=A0=A0=A0 =A0=A0=A0 1
=A0Number of racks:=A0=A0=A0 =A0=A0=A0 1
FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds


The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
[root@sandbox ~]#

Rod Nascimento

Fro= m: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
Sent: Friday, January 24, 2014 3:34 PM
To: user= @hadoop.apache.org
Subject: RE: hdfs fsck -locations

I=92m not seeing location= s flag yet.

=A0

Rod Nas= cimento

Systems= Engineer @ Brazil

=A0=

People don=92t buy WHAT you do. They buy WHY you do it.

=A0

From: Mark Ker= zner [mailto:= mark.kerzner@shmsoft.com]
Sent: Friday, January 24, 2014 3:16 PM
To: Hadoop User
Subject: Re: hdfs fsck -locations

=A0

Sorry, did not copy the full command

=A0

hdfs fsck /user/mark/data/word_count.csv -locations<= /p>

Connecting to namenode via http://mark-7:50070

FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path /user/ma= rk/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014

.Status: HEALTHY

=A0Total size:=A0=A0=A0=A0=A0=A0 7217 B

=A0Total dirs:=A0=A0=A0=A0=A0=A0 0

=A0Total files:=A0=A0=A0=A0=A0 1

=A0Total blocks (validated):=A0=A0=A0=A0=A0=A0=A0 1 = (avg. block size 7217 B)

=A0Minimally replicated blocks:=A0 1 (100.0 %)

=A0Over-replicated blocks:=A0=A0=A0=A0=A0=A0=A0=A0= =A0 0 (0.0 %)

=A0Under-replicated blocks:=A0=A0=A0=A0=A0=A0=A0 0 (= 0.0 %)

=A0Mis-replicated blocks:=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 0 (0.0 %)

=A0Default replication factor:=A0=A0=A0=A0=A0 1

=A0Average block replication:=A0=A0=A0=A0 1.0

=A0Corrupt blocks:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 0

=A0Missing replicas:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 0 (0.0 %)

=A0Number of data-nodes:=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1

=A0Number of racks:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 1

FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 mill= iseconds

=A0

=A0

The filesystem under path '/user/mark/data/word_= count.csv' is HEALTHY

=A0

=A0

On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <harsh@cloudera.com>= ; wrote:

Sorry, but what was the question? I also do not see a locations option f= lag.

On Jan 24, 2014 7:17 PM, "Mark Kerzner" &l= t;mark.kerzne= r@shmsoft.com> wrote:

Here is an example

=A0

=A0hdfs fsck /user/mark/data/word_count.csv

Connecting to namenode via http://mark-7:50070

FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path /user/ma= rk/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014

.Status: HEALTHY

=A0Total size: 7217 B

=A0Total dirs: 0

=A0Total files: 1

=A0Total blocks (validated): 1 (avg. block size 7217= B)

=A0Minimally replicated blocks: 1 (100.0 %)

=A0Over-replicated blocks: 0 (0.0 %)

=A0Under-replicated blocks: 0 (0.0 %)

=A0Mis-replicated blocks: 0 (0.0 %)

=A0Default replication factor: 1

=A0Average block replication: 1.0

=A0Corrupt blocks: 0

=A0Missing replicas: 0 (0.0 %)

=A0Number of data-nodes: 1

=A0Number of racks: 1

FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 mill= iseconds

=A0

=A0

On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <harsh@cloudera.com>= wrote:

Hi Mark,

Yes, the locations are shown as IP.


On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <mark.kerzner@shmsoft.com> wrot= e:
> Hi,
>
> hdfs fsck -locations
>
> is supposed to show every block with its location? Is location the ip = of the
> datanode?
>
> Thank you,
> Mark


--=
Harsh J

=A0

=A0


--f46d0443064e4e158304f0becd49--