hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Obtain the filename that is procesed by Map class when CombineFileInputFormat is used
Date Thu, 28 Jul 2011 14:18:23 GMT
Florin,

I believe you answered yourself accidentally?

On Thu, Jul 28, 2011 at 4:10 PM, Florin P <florinpico@yahoo.com> wrote:
> --- On Fri, 7/22/11, Florin P <florinpico@yahoo.com> wrote:
>
> From: Florin P <florinpico@yahoo.com>
> Subject: Obtain the filename that is procesed by Map class when CombineFileInputFormat
is used
> To: hdfs-user@hadoop.apache.org
> Date: Friday, July 22, 2011, 8:34 AM
>
> Hello!
>   I would like to ask you, how can you obtain the filenames that is processed by Map
class when CombineFileInputFormat is used?
>    As far as I know when using CombineFileInputFormat, multiple files will be processed
by the same mapper. In my case, I would like to know how to obtain these file names.

Depending on your you have implemented your per-FileSplit record
readers in the CFIP, you can set "map.input.file" in the Configuration
instance in each of its initialization. This is somewhat self managed
here since several record readers may be initialized. Let me know if
you would like to see a simple example along as well.

--
Harsh J

Mime
View raw message