Return-Path: 
 <core-user-return-4632-apmail-hadoop-core-user-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-core-user-archive@www.apache.org
Received: (qmail 5358 invoked from network); 20 Feb 2008 02:14:36 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 20 Feb 2008 02:14:36 -0000
Received: (qmail 92123 invoked by uid 500); 20 Feb 2008 02:14:24 -0000
Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org
Received: (qmail 92087 invoked by uid 500); 20 Feb 2008 02:14:24 -0000
Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:core-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:core-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:core-user@hadoop.apache.org>
List-Id: <core-user.hadoop.apache.org>
Reply-To: core-user@hadoop.apache.org
Delivered-To: mailing list core-user@hadoop.apache.org
Received: (qmail 92078 invoked by uid 99); 20 Feb 2008 02:14:24 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Feb 2008 18:14:24 -0800
X-ASF-Spam-Status: No, hits=2.8 required=10.0
	tests=RCVD_IN_DNSWL_LOW,RCVD_NUMERIC_HELO,SPF_NEUTRAL
X-Spam-Check-By: apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Received: from [69.50.2.13] (HELO ex9.myhostedexchange.com) (69.50.2.13)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Feb 2008 02:13:50 +0000
Received: from 206.169.1.36 ([206.169.1.36]) by ex9.hostedexchange.local
 ([69.50.2.13]) with Microsoft Exchange Server HTTP-DAV ;
 Wed, 20 Feb 2008 02:13:58 +0000
User-Agent: Microsoft-Entourage/11.3.3.061214
Date: Tue, 19 Feb 2008 18:13:56 -0800
Subject: Re: FileOutputFormat which does not write key value?
From: Ted Dunning <tdunning@veoh.com>
To: <core-user@hadoop.apache.org>
Message-ID: <C3E0CCE4.38A5B%tdunning@veoh.com>
Thread-Topic: FileOutputFormat which does not write key value?
Thread-Index: AchzZj7yfVofnt9ZEdyM7AAWy8rVfQ==
In-Reply-To: <8e8d18110802191801v10934e69o50ed3ab95e91e3c9@mail.gmail.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit
X-Virus-Checked: Checked by ClamAV on apache.org


Re-reading the thread convinces me that this is a difference between
TextOutputFormat and other output formats.


On 2/19/08 6:01 PM, "Andy Li" <annndy.lee@gmail.com> wrote:

> Shouldn't the official way to do this is to implement your own RecordWriter
> and implement the
> OutputFormatClass.
> 
> conf.setOutputFormat(yourClass);
> 
> Inside the yourClass, you can return your own RecordWriter class in the
> getRecordWriter method.
> 
> I did it on the FileInputFormat with my own RecordReader and it worked for
> me
> to take KEY and null VALUE into the Mapper.  I believe it is the same thing
> vice versa.
> 
> But there should be a formal way instead of try-and-error to see what the
> system default
> is.  I guess the system does not have a standard spec to define what is the
> default values?
> Maybe this is why Ted has such concern of incompatible in 0.16.*?
> 
> -Andy
> 
> On Feb 19, 2008 3:02 PM, Lukas Vlcek <lukas.vlcek@gmail.com> wrote:
> 
>> Hmmm...
>> 
>> May be I should rather go to bet (it is just midnight in my part of the
>> world...) but I think I did what you are saying:
>> 
>> Configuration:
>>         conf.setOutputKeyClass(NullWritable.class);
>>         conf.setOutputValueClass(Text.class);
>> 
>> And the reducer:
>> public class PermutationReduce extends MapReduceBase implements
>> Reducer<Text, Text, NullWritable, Text> {
>> 
>>    public void reduce(Text key, Iterator<Text> values,
>> OutputCollector<NullWritable, Text> output, Reporter reporter) throws
>> IOException {
>>        while (values.hasNext()) {
>>            output.collect(NullWritable.get(), values.next());
>>        }
>> 
>>    }
>> }
>> 
>> Regards,
>> Lukas
>> 
>> On 2/19/08, Owen O'Malley <oom@yahoo-inc.com> wrote:
>>> 
>>> 
>>> On Feb 19, 2008, at 1:52 PM, Lukas Vlcek wrote:
>>> 
>>>> Hi,
>>>> 
>>>> I don't care about key value in the output file. Is there any way
>>>> how I can
>>>> suppress key in the output?
>>>> Is there a way how to tell (Text)OutputFormat not to write key but
>>>> value
>>>> only? Or can I pass my own implementation of RecordWriter into
>>>> FileOutputFormat?
>>> 
>>> The easiest way is to put either null or a NullWritable in for the
>>> key coming out of the reduce. The TextOutputFormat will drop the tab
>>> character. You can also define your own OutputFormat and encode them
>>> as you wish.
>>> 
>>> -- Owen
>>> 
>> 
>> 
>> 
>> --
>> http://blog.lukas-vlcek.com/
>>