hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Isaacson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8615) EOFException in DecompressorStream.java needs to be more verbose
Date Mon, 17 Sep 2012 18:01:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457199#comment-13457199

Andy Isaacson commented on HADOOP-8615:

Thank you for the patch!

bq. Please let me know if there is any procedure for making this tested only on hadoop common
release 0.20.2

Don't worry about the robots testing against the wrong branch, you're not doing anything wrong.

It seems to me this change would be a good thing on trunk as well. Can you port the patch
to trunk?

bq. When the user uses this method and pass the filename, it would be printed in the EOF exception
thrown, if any. So I believe the test cases may not be necessary. I was able to test it locally
by forcefully creating an EOF Exception and verifying the new message as "java.io.EOFException:
Unexpected end of input stream in the file = filename"

I think this should be fairly easy to test -- just write a compressed stream, truncate the
compressed stream, then try to read it, catch the EOFException and verify that the filename
shows up in the exception text.  Or am I missing something?

I'm a little worried about the places where your {{fileName}}-using methods add new default
values, for example:
+  public CompressionInputStream createInputStream(InputStream in, 
+    Decompressor decompressor, String fileName) 
+  throws IOException {
+    return new DecompressorStream(in, decompressor, 
+               conf.getInt("io.file.buffer.size", 4*1024),fileName);
+  }
I'll have to think about it longer, but having a default value of 4k hidden in this method
seems wrong to me at a first glance.  There are a few other instances of this as well.
> EOFException in DecompressorStream.java needs to be more verbose
> ----------------------------------------------------------------
>                 Key: HADOOP-8615
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8615
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.20.2
>            Reporter: Jeff Lord
>              Labels: patch
>         Attachments: HADOOP-8615-release-0.20.2.patch
> In ./src/core/org/apache/hadoop/io/compress/DecompressorStream.java
> The following exception should at least pass back the file that it encounters this error
in relation to:
>   protected void getCompressedData() throws IOException {
>     checkStream();
>     int n = in.read(buffer, 0, buffer.length);
>     if (n == -1) {
>       throw new EOFException("Unexpected end of input stream");
>     }
> This would help greatly to debug bad/corrupt files.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message