hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: fail to get parameters in new API
Date Sat, 07 Aug 2010 07:27:36 GMT
Thank you from Newbie Central!

On Fri, Aug 6, 2010 at 10:49 PM, Harsh J <qwertymaniac@gmail.com> wrote:
> Use this to get the actual path in the New API:
> ((FileSplit) context.getInputSplit()).getPath()
>
> As explained in HADOOP-5973.
>
> On Sat, Aug 7, 2010 at 7:26 AM, Lance Norskog <goksron@gmail.com> wrote:
>> I have the same request. My use case is that I want to do a database
>> join on three CSV files from different files from the DB. So, if I can
>> read the file name, I can deduce which table it is. The map knows the
>> field names from each table file, and maps each file row using the
>> database id as the key. The reducer receives the different sets of
>> fields for the same key and writes out the complete join.
>>
>> Is there any way to find at least the file name, even if not the complete URL?
>>
>> Lance
>>
>> On Fri, Aug 6, 2010 at 7:16 AM, Gang Luo <lgpublic@yahoo.com.cn> wrote:
>>> Hi all,
>>> I want to know which file (path) I am processing in each map task. In the old
>>> API, I can get it by JobConf.get("map.input.file");. When it comes to new API,
>>> the context.getConfiguration.get("map.input.file") returns null. Does that mean
>>> parameter "map.input.file" does not exist in new API? How about other job/task
>>> specific parameters? Is there any documentation talking about this?
>>>
>>> Thanks,
>>> -Gang
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Lance Norskog
>> goksron@gmail.com
>>
>
>
>
> --
> Harsh J
> www.harshj.com
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message