hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dino Kečo <dino.k...@gmail.com>
Subject Re: Provide context to map function
Date Tue, 02 Apr 2013 08:20:58 GMT
You should check multiple input format class which enables you to have more
input formats for same mapper.

Regards,
Dino
On Apr 2, 2013 9:49 AM, "Yanbo Liang" <yanbohappy@gmail.com> wrote:

> protected void map(KEYIN key, VALUEIN value,
>                      Context context) throws IOException,
> InterruptedException {
>     context.write((KEYOUT) key, (VALUEOUT) value);
>   }
>
> Context is a parameter that the execute environment will pass to the map()
> function.
> You can just use it in the map() function.
>
>
> 2013/4/2 Abhinav M Kulkarni <abhinavkulkarni@gmail.com>
>
>>  To be precise, I am using Hadoop 1.0.4.
>>
>> There is no local variable or argument named context in the map function.
>>
>> Thanks,
>> Abhinav
>>
>>
>> On 04/01/2013 09:06 PM, Azuryy Yu wrote:
>>
>>   I supposed your input splits are FileSplit, if not, you need to:
>>
>>  InputSplit split = context.getInputSplit();
>>
>>  if (split instanceof FileSplit){
>>    Path path = ((FileSplit)split).getPath();
>> }
>>
>>
>>
>>
>> On Tue, Apr 2, 2013 at 12:02 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>>
>>>  In your map function add following:
>>>
>>> Path currentInput = ((FileSplit)context.getInputSplit()).getPath();
>>>
>>>  then:
>>>
>>>  if (currentInput is first ){
>>> ................
>>> }
>>>  else{
>>> ..................
>>> }
>>>
>>>
>>>
>>>
>>> On Tue, Apr 2, 2013 at 11:55 AM, Abhinav M Kulkarni <
>>> abhinavkulkarni@gmail.com> wrote:
>>>
>>>>  Hi,
>>>>
>>>> I have a following scenario:
>>>>
>>>>
>>>>    - Two mappers (acting on two different files) and one reducer
>>>>    - The mapper code for two different files is the same, except for
>>>>    minor change which depends on which file is being read
>>>>    - Essentially assume there is an if statement - if first file is
>>>>    being read do this else do this
>>>>
>>>> So how do I provide this context to map function i.e. file name or say
>>>> a boolean flag variable indicating the file being read?
>>>>
>>>> Thanks,
>>>> Abhinav
>>>>
>>>
>>>
>>
>>
>

Mime
View raw message