hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: fail to get parameters in new API
Date Sat, 07 Aug 2010 05:49:10 GMT
Use this to get the actual path in the New API:
((FileSplit) context.getInputSplit()).getPath()

As explained in HADOOP-5973.

On Sat, Aug 7, 2010 at 7:26 AM, Lance Norskog <goksron@gmail.com> wrote:
> I have the same request. My use case is that I want to do a database
> join on three CSV files from different files from the DB. So, if I can
> read the file name, I can deduce which table it is. The map knows the
> field names from each table file, and maps each file row using the
> database id as the key. The reducer receives the different sets of
> fields for the same key and writes out the complete join.
> Is there any way to find at least the file name, even if not the complete URL?
> Lance
> On Fri, Aug 6, 2010 at 7:16 AM, Gang Luo <lgpublic@yahoo.com.cn> wrote:
>> Hi all,
>> I want to know which file (path) I am processing in each map task. In the old
>> API, I can get it by JobConf.get("map.input.file");. When it comes to new API,
>> the context.getConfiguration.get("map.input.file") returns null. Does that mean
>> parameter "map.input.file" does not exist in new API? How about other job/task
>> specific parameters? Is there any documentation talking about this?
>> Thanks,
>> -Gang
> --
> Lance Norskog
> goksron@gmail.com

Harsh J

View raw message