hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: fail to get parameters in new API
Date Sat, 07 Aug 2010 01:56:58 GMT
I have the same request. My use case is that I want to do a database
join on three CSV files from different files from the DB. So, if I can
read the file name, I can deduce which table it is. The map knows the
field names from each table file, and maps each file row using the
database id as the key. The reducer receives the different sets of
fields for the same key and writes out the complete join.

Is there any way to find at least the file name, even if not the complete URL?


On Fri, Aug 6, 2010 at 7:16 AM, Gang Luo <lgpublic@yahoo.com.cn> wrote:
> Hi all,
> I want to know which file (path) I am processing in each map task. In the old
> API, I can get it by JobConf.get("map.input.file");. When it comes to new API,
> the context.getConfiguration.get("map.input.file") returns null. Does that mean
> parameter "map.input.file" does not exist in new API? How about other job/task
> specific parameters? Is there any documentation talking about this?
> Thanks,
> -Gang

Lance Norskog

View raw message