hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Compressing map only output
Date Tue, 01 May 2012 03:22:46 GMT
Hey Mohit,

Most of what you need to know for jobs is available at
http://hadoop.apache.org/common/docs/current/mapred_tutorial.html

A more complete, mostly unseparated list of config params are also
available at: http://hadoop.apache.org/common/docs/current/mapred-default.html
(core-default.html, hdfs-default.html)

On Tue, May 1, 2012 at 6:36 AM, Mohit Anchlia <mohitanchlia@gmail.com> wrote:
> Thanks! When I tried to search for this property I couldn't find it. Is
> there a page that has complete list of properties and it's usage?
>
> On Mon, Apr 30, 2012 at 5:44 PM, Prashant Kommireddi <prash1784@gmail.com>wrote:
>
>> Yes. These are hadoop properties - using "set" is just a way for Pig to set
>> those properties in your job conf.
>>
>>
>> On Mon, Apr 30, 2012 at 5:25 PM, Mohit Anchlia <mohitanchlia@gmail.com
>> >wrote:
>>
>> > Is there a way to compress map only jobs to compress map output that gets
>> > stored on hdfs as part-m-* files? In pig I used :
>> >
>> > Would these work form plain map reduce jobs as well?
>> >
>> >
>> > set output.compression.enabled true;
>> >
>> > set output.compression.codec org.apache.hadoop.io.compress.SnappyCodec;
>> >
>>



-- 
Harsh J

Mime
View raw message