chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CHUKWA-1) Switch lzo to other compression algorithm for Chukwa Records
Date Wed, 01 Apr 2009 22:56:13 GMT

     [ https://issues.apache.org/jira/browse/CHUKWA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jerome Boulon updated CHUKWA-1:
-------------------------------

    Status: Patch Available  (was: Open)

To be compliant with Apache license we have to remove all references to LZO, however you can
still use LZO and achieve the same behavior by adding the lines below to your hadoop-site.xml
or chukwa-demux-conf.xml.

<property>
  <name>mapred.output.compress</name>
  <value>true</value>
  <description>Should the job outputs be compressed?
  </description>
</property>

<property>
  <name>mapred.output.compression.type</name>
  <value>BLOCK</value>
  <description>If the job outputs are to compressed as SequenceFiles, how should
               they be compressed? Should be one of NONE, RECORD or BLOCK.
  </description>
</property>

<property>
  <name>mapred.output.compression.codec</name>
  <value>org.apache.hadoop.io.compress.LzoCodec</value>
  <description>If the job outputs are compressed, how should they be compressed?
  </description>
</property>

<property>
  <name>mapred.compress.map.output</name>
  <value>true</value>
  <description>Should the outputs of the maps be compressed before being
               sent across the network. Uses SequenceFile compression.
  </description>
</property>

<property>
  <name>mapred.map.output.compression.codec</name>
  <value>org.apache.hadoop.io.compress.LzoCodec</value>
  <description>If the map outputs are compressed, how should they be
               compressed?
  </description>
</property>


> Switch lzo to other compression algorithm for Chukwa Records
> ------------------------------------------------------------
>
>                 Key: CHUKWA-1
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-1
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>         Environment: Redhat 5.1, Java 6
>            Reporter: Eric Yang
>            Assignee: Jerome Boulon
>            Priority: Blocker
>         Attachments: CHUKWA-1.patch
>
>
> Chukwa Records depends on the lzo library from Hadoop.  Hadoop is removing lzo, therefore
Chukwa needs to switch to another compression algorithm.  This jira is to keep track of the
shell script and java file changes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message