Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 43F4C79F3 for ; Thu, 28 Jul 2011 14:19:13 +0000 (UTC) Received: (qmail 232 invoked by uid 500); 28 Jul 2011 14:19:12 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 141 invoked by uid 500); 28 Jul 2011 14:19:11 -0000 Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-user@hadoop.apache.org Delivered-To: mailing list hdfs-user@hadoop.apache.org Received: (qmail 132 invoked by uid 99); 28 Jul 2011 14:19:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jul 2011 14:19:11 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates 74.125.83.48 as permitted sender) Received: from [74.125.83.48] (HELO mail-gw0-f48.google.com) (74.125.83.48) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jul 2011 14:19:04 +0000 Received: by gwj22 with SMTP id 22so2809213gwj.35 for ; Thu, 28 Jul 2011 07:18:43 -0700 (PDT) Received: by 10.236.190.68 with SMTP id d44mr41259yhn.393.1311862723082; Thu, 28 Jul 2011 07:18:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.236.60.163 with HTTP; Thu, 28 Jul 2011 07:18:23 -0700 (PDT) In-Reply-To: <1311849652.2056.YahooMailClassic@web29216.mail.ird.yahoo.com> References: <1311338086.96751.YahooMailClassic@web29201.mail.ird.yahoo.com> <1311849652.2056.YahooMailClassic@web29216.mail.ird.yahoo.com> From: Harsh J Date: Thu, 28 Jul 2011 19:48:23 +0530 Message-ID: Subject: Re: Obtain the filename that is procesed by Map class when CombineFileInputFormat is used To: hdfs-user@hadoop.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Florin, I believe you answered yourself accidentally? On Thu, Jul 28, 2011 at 4:10 PM, Florin P wrote: > --- On Fri, 7/22/11, Florin P wrote: > > From: Florin P > Subject: Obtain the filename that is procesed by Map class when CombineFi= leInputFormat is used > To: hdfs-user@hadoop.apache.org > Date: Friday, July 22, 2011, 8:34 AM > > Hello! > =A0 I would like to ask you, how can you obtain the filenames that is pro= cessed by Map class when CombineFileInputFormat is used? > =A0=A0 As far as I know when using CombineFileInputFormat, multiple files= will be processed by the same mapper. In my case, I would like to know how= to obtain these file names. Depending on your you have implemented your per-FileSplit record readers in the CFIP, you can set "map.input.file" in the Configuration instance in each of its initialization. This is somewhat self managed here since several record readers may be initialized. Let me know if you would like to see a simple example along as well. -- Harsh J