hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemanth Yamijala <yhema...@gmail.com>
Subject Re: Intermediate files generated.
Date Fri, 02 Jul 2010 01:52:13 GMT
Alex,

> I don't think this is what I am looking for. Essential, I wish to run both
> mapper as well as reducer. But at the same time, i wish to make sure that
> the temp files that are used between mappers and reducers are of my choice.
> Here, the choice means that I can specify the files in HDFS that can be used
> as temp files.

Could you explain why you want to do this ?

>
> thanks,
> --PB.
>
> On Fri, Jul 2, 2010 at 12:14 AM, Alex Loddengaard <alex@cloudera.com> wrote:
>
>> You could use the HDFS API from within your mapper, and run with 0
>> reducers.
>>
>> Alex
>>
>> On Thu, Jul 1, 2010 at 3:07 PM, Pramy Bhats <pramybhats@googlemail.com
>> >wrote:
>>
>> > Hi,
>> >
>> > I am using hadoop framework for writing MapReduce jobs. I want  to
>> redirect
>> > the output of Map into files of my choice and later use those files as
>> > input
>> > for Reduce phase.
>> >
>> >
>> > Could you please suggest, how to proceed for it ?
>> >
>> > thanks,
>> > --PB.
>> >
>>
>

Mime
View raw message