hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Lyubimov <dlie...@gmail.com>
Subject Re: MultipleOutputFormat
Date Wed, 30 Mar 2011 04:13:15 GMT
Oh. ok, let me look at it. thanks a bunch.

On Tue, Mar 29, 2011 at 9:09 PM, Alejandro Abdelnur <tucu@cloudera.com> wrote:
> You should be able to create partitions on the fly.
> Check the last example in the javadocs:
>
> http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html
>
>  ...
>
>   mos.write(key, new Text("value"), generateFileName(key, new
> Text("value")));
>
> Hope this helps.
> Alejandro
> On Wed, Mar 30, 2011 at 12:02 PM, Dmitriy Lyubimov <dlieu.7@gmail.com>
> wrote:
>>
>> yes.. but in my old code the file names are created on the fly (it
>> basically creates partitions based on a time field). I dont think
>> MultipleOutputs is not suitable to create partitions on the fly.
>>
>> On Tue, Mar 29, 2011 at 8:56 PM, Alejandro Abdelnur <tucu@cloudera.com>
>> wrote:
>> > Dmitriy,
>> > Have you check the MultipleOutputs instead? It provides similar
>> > functionality.
>> > Alejandro
>> >
>> > On Wed, Mar 30, 2011 at 11:39 AM, Dmitriy Lyubimov <dlieu.7@gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >> I can't seem to be able to find either jira or implementation of
>> >> MultipleOutputFormat in new api in either 0.21 or 0.22 branches.
>> >> Are there any plans to port that to new api as well?
>> >>
>> >> thanks in advance.
>> >> -Dmitriy
>> >
>> >
>
>

Mime
View raw message