Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2760E112D1 for ; Fri, 28 Mar 2014 17:30:34 +0000 (UTC) Received: (qmail 59904 invoked by uid 500); 28 Mar 2014 17:30:23 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 59715 invoked by uid 500); 28 Mar 2014 17:30:16 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 59672 invoked by uid 99); 28 Mar 2014 17:30:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Mar 2014 17:30:13 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of smarty.juice@gmail.com designates 209.85.219.54 as permitted sender) Received: from [209.85.219.54] (HELO mail-oa0-f54.google.com) (209.85.219.54) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Mar 2014 17:30:07 +0000 Received: by mail-oa0-f54.google.com with SMTP id n16so6476564oag.13 for ; Fri, 28 Mar 2014 10:29:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=fso4euLxyOgEceDRyGjFIvZCR/mOfMPE9nkU/v7Qw0E=; b=HIj1p0QrtiNdcTXtR+Bkjil1qZpnsOChfP1tSAX6OrRqJj1QkCfeVF9YOAftHMGaj6 Ecp/oy0pSG6gBD4h5RCvNxyMMw0TekLmHIUJOsUHXC6UxJPJz3A8PGyRHwHj8QwfZGkE jTlYDR60OoZUrEUr6nkvBQe9J1EXZgYMD/ImDsITSShFGubeFIhrh17FigRFFqZHCMvJ 6LR5NckIEtmxfRLxJQJKMNkELdXUimmG6RDh+4+TPmnLa+xVujNvcAPQI3o36S0dn/vx ACTUNKE/5meG/KV3kVS0aagcZgTE/T7jzD3x7uEm+AvsYSPmJiVOJEIB/hnQikWoij0v mjwQ== MIME-Version: 1.0 X-Received: by 10.182.28.7 with SMTP id x7mr7783091obg.43.1396027784998; Fri, 28 Mar 2014 10:29:44 -0700 (PDT) Received: by 10.182.28.165 with HTTP; Fri, 28 Mar 2014 10:29:44 -0700 (PDT) In-Reply-To: References: Date: Fri, 28 Mar 2014 13:29:44 -0400 Message-ID: Subject: Re: How to get locations of blocks programmatically? From: Hardik Pandya To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=089e015380ba38b68d04f5ae0c1e X-Virus-Checked: Checked by ClamAV on apache.org --089e015380ba38b68d04f5ae0c1e Content-Type: text/plain; charset=ISO-8859-1 have you looked into FileSystem API this is hadoop v2.2.0 http://hadoop.apache.org/docs/r2.2.0/api/org/apache/hadoop/fs/FileSystem.html does not exist in http://hadoop.apache.org/docs/r1.2.0/api/org/apache/hadoop/fs/FileSystem.html org.apache.hadoop.fs.RemoteIterator > *listFiles * (Path f, boolean recursive) List the statuses and block locations of the files in the given path. org.apache.hadoop.fs.RemoteIterator > *listLocatedStatus * (Path f) List the statuses of the files/directories in the given path if the path is a directory. On Thu, Mar 27, 2014 at 10:03 PM, Libo Yu wrote: > Hi all, > > "hadoop path fsck -files -block -locations" can list locations for all > blocks in the path. > Is it possible to list all blocks and the block locations for a given path > programmatically? > Thanks, > > Libo > --089e015380ba38b68d04f5ae0c1e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
have you looked into FileSystem API this is hadoop v2= .2.0

http://hadoop.apache.org/docs/r2.2.0/api/org/ap= ache/hadoop/fs/FileSystem.html

does not exist in http://hadoop.apache.org/docs/r1= .2.0/api/org/apache/hadoop/fs/FileSystem.html

=A0org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus><= /td> listFiles(Pat= h=A0f, boolean=A0recursive)
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0List the statuses and block locations of the = files in the given path.
=A0org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus><= /td>

On Thu, Mar 27, 2014 at 10:03 PM,= Libo Yu <yu_libo@hotmail.com> wrote:
Hi all,

"hadoop path fsck -files -block -= locations" can list locations for all blocks in the path.
Is it pos= sible to list all blocks and the block locations for a given path programma= tically?
Thanks,

Libo

--089e015380ba38b68d04f5ae0c1e--