Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Wed, 24 Oct 2012 01:10:12 +0000 (UTC)
From: "Ivan Mitic (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <1987533755.19584.1351041012517.JavaMail.jiratomcat@arcas>
In-Reply-To: <441339784.68031.1340907284585.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Resolved] (HADOOP-8540) Compression with non-default
 codec's fail on Windows
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Mitic resolved HADOOP-8540.
--------------------------------

    Resolution: Fixed

Support for zlib compression library was added with HADOOP-8564. 

To enable zlib compression codecs on Windows:
 - Download zlib sources and set ZLIB_HOME in the environment before building Hadoop on Windows (this step is needed for build to generate the corresponding stubs in hadoop.dll)
 - Install zlib1.dll to system32 (with HADOOP-8907 you can also install it next to hadoop.dll)
                
> Compression with non-default codec's fail on Windows
> ----------------------------------------------------
>
>                 Key: HADOOP-8540
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8540
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Trupti Dhavle
>
> I was trying to run the compression test case with GZipCodec. However the test fails with NPE. It is running fine with DefaultCodec.
> With text input
> c:\Workspace\winhadoopqe>c:\hdp\branch-1-win\bin\hadoop jar c:\hdp\branch-1-win\
> build\hadoop-examples-1.1.0-SNAPSHOT.jar sort "-Dmapred.compress.map.output=true
> " "-Dmapred.map.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec
> " "-Dmapred.output.compress=true"  "-Dmapred.output.compression.type=NONE"  "-Dm
> apred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec" -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text  Compression/texti
> nput Compression/textoutput-123
> Running on 1 nodes to sort from hdfs://localhost:8020/user/Administrator/Compres
> sion/textinput into hdfs://localhost:8020/user/Administrator/Compression/textout
> put-123 with 1 reduces.
> Job started: Thu Jun 28 10:02:18 PDT 2012
> 12/06/28 10:02:18 INFO mapred.FileInputFormat: Total input paths to process : 1
> 12/06/28 10:02:18 INFO mapred.JobClient: Running job: job_201206271409_0045
> 12/06/28 10:02:19 INFO mapred.JobClient:  map 0% reduce 0%
> 12/06/28 10:02:36 INFO mapred.JobClient: Task Id : attempt_201206271409_0045_m_0
> 00000_0, Status : FAILED
> java.lang.NullPointerException
>         at org.apache.hadoop.mapred.IFile$Writer.<init>(IFile.java:102)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask
> .java:1407)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1
> 298)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:437)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:271)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
> tion.java:1103)
>         at org.apache.hadoop.mapred.Child.main(Child.java:265)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira