hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 皮皮 <pi.bingf...@gmail.com>
Subject Re: Multiple k,v pairs from a single map - possible?
Date Tue, 07 Apr 2009 07:29:32 GMT
could any body tell me how to get one of the multipleoutput file in another
jobconfig?

2009/4/3 皮皮 <pi.bingfeng@gmail.com>

> thank you very much . this is what i am looking for.
>
> 2009/3/27 Brian MacKay <Brian.MacKay@medecision.com>
>
>
>> Amandeep,
>>
>> Add this to your driver.....
>>
>> MultipleOutputs.addNamedOutput(conf, "PHONE",TextOutputFormat.class,
>> Text.class, Text.class);
>>
>> MultipleOutputs.addNamedOutput(conf, "NAME,
>>                    TextOutputFormat.class, Text.class, Text.class);
>>
>>
>>
>> And in your reducer....
>>
>>  private MultipleOutputs mos;
>>
>> public void reduce(Text key, Iterator<Text> values,
>>            OutputCollector<Text, Text> output, Reporter reporter) {
>>
>>
>>          // namedOutPut = either PHONE or NAME
>>
>>        while (values.hasNext()) {
>>            String value = values.next().toString();
>>            mos.getCollector(namedOutPut, reporter).collect(
>>                    new Text(value), new Text(othervals));
>>        }
>>    }
>>
>>    @Override
>>    public void configure(JobConf conf) {
>>        super.configure(conf);
>>        mos = new MultipleOutputs(conf);
>>    }
>>
>>    public void close() throws IOException {
>>        mos.close();
>>    }
>>
>>
>>
>> By the way, have you had a change to post your Oracle fix to
>> DBInputFormat ?
>> If so, what is the Jira tag #?
>>
>> Brian
>>
>> -----Original Message-----
>> From: Amandeep Khurana [mailto:amansk@gmail.com]
>> Sent: Friday, March 27, 2009 5:46 AM
>> To: core-user@hadoop.apache.org
>> Subject: Multiple k,v pairs from a single map - possible?
>>
>> Is it possible to output multiple key value pairs from a single map
>> function
>> run?
>>
>> For example, the mapper outputing <name,phone> and <name, address>
>> simultaneously...
>>
>> Can I write multiple output.collect(...) commands?
>>
>> Amandeep
>>
>> Amandeep Khurana
>> Computer Science Graduate Student
>> University of California, Santa Cruz
>>
>>
>>
>>
>>
>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>> _
>>
>> The information transmitted is intended only for the person or entity to
>> which it is addressed and may contain confidential and/or privileged
>> material. Any review, retransmission, dissemination or other use of, or
>> taking of any action in reliance upon, this information by persons or
>> entities other than the intended recipient is prohibited. If you received
>> this message in error, please contact the sender and delete the material
>> from any computer.
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message