Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of stan.ieugen@gmail.com
 designates 74.125.82.48 as permitted sender)
Message-ID: <4F264E4B.9030003@gmail.com>
Date: Mon, 30 Jan 2012 10:01:15 +0200
From: Ioan Eugen Stan <stan.ieugen@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
 rv:1.9.2.24) Gecko/20111108 Lightning/1.0b2 Thunderbird/3.1.16
MIME-Version: 1.0
To: mapreduce-user@hadoop.apache.org
Subject: Re: Fw: reducers outputs
References: <1327817102.96189.YahooMailNeo@web39702.mail.mud.yahoo.com>
 <CAEGKEUMoJ-Mc+B4UQfO_Zv7-BaB3AkLwdxoSBxohRea9T+0qwA@mail.gmail.com>
 <1327901934.94910.YahooMailNeo@web39708.mail.mud.yahoo.com>
 <1327902441.59390.YahooMailNeo@web39702.mail.mud.yahoo.com>
In-Reply-To: <1327902441.59390.YahooMailNeo@web39702.mail.mud.yahoo.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Pe 30.01.2012 07:47, aliyeh saeedi a scris:
>
>
>
>
>
>
> I want to save them with my own names, How NameNode will keep their names?
>
>
>
> ________________________________
>   From: Joey Echeverria<joey@cloudera.com>
> To: mapreduce-user@hadoop.apache.org; aliyeh saeedi<a1_saeedi@yahoo.com>
> Sent: Sunday, 29 January 2012, 17:10
> Subject: Re: reducers outputs
>
> Reduce output is normally stored in HDFS, just like your other files.
> Are you seeing
>   different behavior?
>
> -Joey
>
> On Sun, Jan 29, 2012 at 1:05 AM, aliyeh saeedi<a1_saeedi@yahoo.com>  wrote:
>> Hi
>> I want to save reducers outputs like other files in Hadoop. Does NameNode
>> keep any information about them? How can I do this?
>> Or can I add a new component to Hadoop like NameNode and make JobTracker to
>> consult with it too (I mean I want to make JobTracker to consult with
>> NameNode AND myNewComponent both)?
>
>
>

You aren't making a lot of sens, at least to me :). But if tou wish to 
save reducer output somehow different you will have to implement your 
own class that implements 
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/OutputFormat.html. 
It's easier to extend 
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/FileOutputFormat.html 
and override the parts that you need.

The framework will call methods from the above mentioned class to 
persist the data from reducers. You instruct the framework to use your 
class when you call 
job.setOutputFormatClass(SequenceFileOutputFormat.class); (this makes 
the output a SequenceFile).

Example to save under a different name:

public static class RenamedSequenceFile extends SequenceFileOutputFormat {

         @Override
         public Path getDefaultWorkFile(TaskAttemptContext context, 
String extension) throws IOException {
             FileOutputCommitter committer = (FileOutputCommitter) 
getOutputCommitter(context);
             return new Path(committer.getWorkPath(), "myBetterName");
         }
     }

This will output your reducer data into "myBetterName" file as key 
values pairs (behaviour inherited from SequanceFileOutputFormat).

I hope this helps,

-- 
Ioan Eugen Stan
http://ieugen.blogspot.com