Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 79E0210C7C for ; Wed, 11 Dec 2013 07:17:49 +0000 (UTC) Received: (qmail 74042 invoked by uid 500); 11 Dec 2013 07:17:42 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 73756 invoked by uid 500); 11 Dec 2013 07:17:41 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 73749 invoked by uid 99); 11 Dec 2013 07:17:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Dec 2013 07:17:41 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates 209.85.223.174 as permitted sender) Received: from [209.85.223.174] (HELO mail-ie0-f174.google.com) (209.85.223.174) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Dec 2013 07:17:36 +0000 Received: by mail-ie0-f174.google.com with SMTP id at1so10412482iec.33 for ; Tue, 10 Dec 2013 23:17:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=zoflkK1gHaVenJfJNIULuBLPt4904zmclWkUFxwdNMo=; b=cbpjXL8xtIkOKeZrJ21sf94u3jKJmlhfxft3qo+2wjmlPjtWRgjfxl51YWA2sPC3DH w1lz/OX6IWGcfSgZKO/uJ5Ld8MWov2vQz4O4HiJwtTnACLT3co2udpm4YM6oF11iulzW 3wAOqnzsKHfuKGASGDB8cvrv2l6M83hem1zOnzi/ktS3bWfyV2x3czbOpeC1nUHjZKVW sYPicovpppYXfJ3WrUAmsXyrHAhgsvRZI83b1Pg8GasS2ApuxgvDk+gI/sXiyEDWPzIt Q0zj2NkCHAsIfl8V1MFrOrbO+Pd3MXCd2bS6QmLAXIgSXdGlpMXvA2asUOJ2M7/ajX3+ OauQ== X-Gm-Message-State: ALoCoQk5AjRuULT5ULvG/BPlwofM4XlRA3Dp+P7Ktl6r8gqPd0cD161eND1y4vcRROFawNfXbl+8 X-Received: by 10.50.143.10 with SMTP id sa10mr1544226igb.8.1386746235424; Tue, 10 Dec 2013 23:17:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.50.234.225 with HTTP; Tue, 10 Dec 2013 23:16:55 -0800 (PST) In-Reply-To: References: From: Harsh J Date: Wed, 11 Dec 2013 12:46:55 +0530 Message-ID: Subject: Re: issue about corrupt block test To: "" Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org Block files are not stored in a flat directory (to avoid FS limits of max files under a dir). Instead of looking for them right under finalized, issue a "find" query with the pattern instead and you should be able to spot it. On Wed, Dec 11, 2013 at 9:10 AM, ch huang wrote: > hi,maillist: > i try to corrupt a block of a file in my benchmark environment, > as the following command i find blk_2504407693800874616_106252 ,it's replica > on 192.168.10.224 is my target ,but i find all the datadir in 192.168.10.224 > ,can not fine the datafile belongs to this replic ,why? > > # ls > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232* > ls: cannot access > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*: > No such file or directory > [root@CHBM224 conf]# ls > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252* > ls: cannot access > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*: > No such file or directory > [root@CHBM224 conf]# ls > /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252* > ls: cannot access > /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*: > No such file or directory > [root@CHBM224 conf]# ls > /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252* > ls: cannot access > /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*: > No such file or directory > [root@CHBM224 conf]# hdfs fsck /alex/terasort/1G-input/part-m-00000 -files > -blocks -locations > Connecting to namenode via http://CHBM220:50070 > FSCK started by root (auth:SIMPLE) from /192.168.10.224 for path > /alex/terasort/1G-input/part-m-00000 at Wed Dec 11 11:35:42 CST 2013 > /alex/terasort/1G-input/part-m-00000 100000000 bytes, 2 block(s): OK > 0. BP-50684181-192.168.10.220-1383638483950:blk_3717620888497075523_106232 > len=67108864 repl=3 [192.168.10.222:50010, 192.168.10.223:50010, > 192.168.10.221:50010] > 1. BP-50684181-192.168.10.220-1383638483950:blk_2504407693800874616_106252 > len=32891136 repl=3 [192.168.10.222:50010, 192.168.10.221:50010, > 192.168.10.224:50010] -- Harsh J