hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <mark.kerz...@shmsoft.com>
Subject Re: hdfs fsck -locations
Date Sun, 26 Jan 2014 01:11:35 GMT
Got it, Harsh, now I see it all.

Thank you,
Mark


On Fri, Jan 24, 2014 at 7:49 PM, Harsh J <harsh@cloudera.com> wrote:

> The right syntax is to use "-files -blocks -locations", so it drills
> down all the way. You are not "missing" a feature - this has existed
> for as long as I've known HDFS.
>
> In Rodrigo's output, he's seeing a BlockPool ID, which is not
> equivalent to a location, but just carries an IP in it for
> identification purposes.
>
> On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <mark.kerzner@shmsoft.com>
> wrote:
> > Yes, Rodrigo,
> >
> > that's what I was looking for. So in my install I somehow don't have it
> at
> > all. Was asked by my students, so I got the answer.
> >
> > Mark
> >
> >
> > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo
> > <Rodrigo.Nascimento@netapp.com> wrote:
> >>
> >> Mark,
> >>
> >> there we go ;-)
> >>
> >> Rodrigo Nascimento
> >> Systems Engineer @ Brazil
> >> Mobile +55 11 991.873.810
> >>
> >> Sent from my iPhone
> >>
> >> Begin forwarded message:
> >>
> >> From: "Nascimento, Rodrigo" <Rodrigo.Nascimento@netapp.com>
> >> Date: 24 de janeiro de 2014 15:59:33 BRST
> >> To: "user@hadoop.apache.org" <user@hadoop.apache.org>
> >> Subject: RE: hdfs fsck -locations
> >>
> >> Hi Mark,
> >>
> >> It is a sample from my sandbox. Your question is about the part that is
> in
> >> RED at the output below, right?
> >>
> >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd  -locations
> >> Connecting to namenode via http://sandbox.hortonworks.com:50070
> >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path
> >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014
> >> .
> >> /user/ambari-qa/passwd:  Under replicated
> >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target
> Replicas
> >> is 3 but found 1 replica(s).
> >> Status: HEALTHY
> >>  Total size:    1708 B
> >>  Total dirs:    0
> >>  Total files:    1
> >>  Total symlinks:        0
> >>  Total blocks (validated):    1 (avg. block size 1708 B)
> >>  Minimally replicated blocks:    1 (100.0 %)
> >>  Over-replicated blocks:    0 (0.0 %)
> >>  Under-replicated blocks:    1 (100.0 %)
> >>  Mis-replicated blocks:        0 (0.0 %)
> >>  Default replication factor:    3
> >>  Average block replication:    1.0
> >>  Corrupt blocks:        0
> >>  Missing replicas:        2 (66.666664 %)
> >>  Number of data-nodes:        1
> >>  Number of racks:        1
> >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds
> >>
> >>
> >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY
> >> [root@sandbox ~]#
> >>
> >> Rod Nascimento
> >>
> >> ________________________________
> >> From: Nascimento, Rodrigo [Rodrigo.Nascimento@netapp.com]
> >> Sent: Friday, January 24, 2014 3:34 PM
> >> To: user@hadoop.apache.org
> >> Subject: RE: hdfs fsck -locations
> >>
> >> I’m not seeing locations flag yet.
> >>
> >>
> >>
> >> Rod Nascimento
> >>
> >> Systems Engineer @ Brazil
> >>
> >>
> >>
> >> People don’t buy WHAT you do. They buy WHY you do it.
> >>
> >>
> >>
> >> From: Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> >> Sent: Friday, January 24, 2014 3:16 PM
> >> To: Hadoop User
> >> Subject: Re: hdfs fsck -locations
> >>
> >>
> >>
> >> Sorry, did not copy the full command
> >>
> >>
> >>
> >> hdfs fsck /user/mark/data/word_count.csv -locations
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >>  Total size:       7217 B
> >>
> >>  Total dirs:       0
> >>
> >>  Total files:      1
> >>
> >>  Total blocks (validated):        1 (avg. block size 7217 B)
> >>
> >>  Minimally replicated blocks:  1 (100.0 %)
> >>
> >>  Over-replicated blocks:          0 (0.0 %)
> >>
> >>  Under-replicated blocks:        0 (0.0 %)
> >>
> >>  Mis-replicated blocks:                        0 (0.0 %)
> >>
> >>  Default replication factor:      1
> >>
> >>  Average block replication:     1.0
> >>
> >>  Corrupt blocks:                      0
> >>
> >>  Missing replicas:                     0 (0.0 %)
> >>
> >>  Number of data-nodes:                      1
> >>
> >>  Number of racks:                   1
> >>
> >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <harsh@cloudera.com> wrote:
> >>
> >> Sorry, but what was the question? I also do not see a locations option
> >> flag.
> >>
> >> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <mark.kerzner@shmsoft.com>
> wrote:
> >>
> >> Here is an example
> >>
> >>
> >>
> >>  hdfs fsck /user/mark/data/word_count.csv
> >>
> >> Connecting to namenode via http://mark-7:50070
> >>
> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path
> >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014
> >>
> >> .Status: HEALTHY
> >>
> >>  Total size: 7217 B
> >>
> >>  Total dirs: 0
> >>
> >>  Total files: 1
> >>
> >>  Total blocks (validated): 1 (avg. block size 7217 B)
> >>
> >>  Minimally replicated blocks: 1 (100.0 %)
> >>
> >>  Over-replicated blocks: 0 (0.0 %)
> >>
> >>  Under-replicated blocks: 0 (0.0 %)
> >>
> >>  Mis-replicated blocks: 0 (0.0 %)
> >>
> >>  Default replication factor: 1
> >>
> >>  Average block replication: 1.0
> >>
> >>  Corrupt blocks: 0
> >>
> >>  Missing replicas: 0 (0.0 %)
> >>
> >>  Number of data-nodes: 1
> >>
> >>  Number of racks: 1
> >>
> >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <harsh@cloudera.com> wrote:
> >>
> >> Hi Mark,
> >>
> >> Yes, the locations are shown as IP.
> >>
> >>
> >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner <
> mark.kerzner@shmsoft.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > hdfs fsck -locations
> >> >
> >> > is supposed to show every block with its location? Is location the ip
> of
> >> > the
> >> > datanode?
> >> >
> >> > Thank you,
> >> > Mark
> >>
> >>
> >> --
> >> Harsh J
> >>
> >>
> >>
> >>
> >
> >
>
>
>
> --
> Harsh J
>

Mime
View raw message