hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5281) GzipCodec fails second time it is used in a process
Date Tue, 17 Mar 2009 01:23:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Douglas updated HADOOP-5281:
----------------------------------

    Attachment: 5281-0.patch
                GZt.java

When GzipCodec is specified, it's used only for the data, not the index (which always uses
DefaultCodec). When the two match, there's no problem.

It's clear that the index is being read with GzipCodec after being written with DefaultCodec
(so it expects a header, but doesn't find one). Still trying to determine how that's happening...

This didn't work with previous versions of Hadoop, right?

> GzipCodec fails second time it is used in a process
> ---------------------------------------------------
>
>                 Key: HADOOP-5281
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5281
>             Project: Hadoop Core
>          Issue Type: Bug
>    Affects Versions: 0.19.0
>         Environment: 0.19.0 / Linux / amd64
>            Reporter: Ben Maurer
>            Priority: Blocker
>         Attachments: 5281-0.patch, GZt.java, GZt.java
>
>
> The attached code (GZt.java) raises:
> {noformat}
> java.io.IOException: incorrect header check
> 	at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native Method)
> 	at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:221)
> 	at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:80)
> 	at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:74)
> 	at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:62)
> 	at java.io.DataInputStream.readByte(DataInputStream.java:248)
> 	at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:325)
> 	at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:346)
> 	at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1853)
> 	at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> 	at org.apache.hadoop.io.MapFile$Reader.readIndex(MapFile.java:319)
> 	at org.apache.hadoop.io.MapFile$Reader.seekInternal(MapFile.java:435)
> 	at org.apache.hadoop.io.MapFile$Reader.seekInternal(MapFile.java:417)
> 	at org.apache.hadoop.io.MapFile$Reader.seek(MapFile.java:404)
> 	at org.apache.hadoop.io.MapFile$Reader.get(MapFile.java:523)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message