commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter De Maeyer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (COMPRESS-206) TarArchiveOutputStream sometimes writes garbage beyond the end of the archive
Date Wed, 24 Oct 2012 12:52:12 GMT

    [ https://issues.apache.org/jira/browse/COMPRESS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483185#comment-13483185
] 

Peter De Maeyer edited comment on COMPRESS-206 at 10/24/12 12:51 PM:
---------------------------------------------------------------------

I think the root cause is that TarArchiveInputStream stops reading the input stream once it
hits the first EOF record. Any lingering records after that are left unconsumed. It would
be best to consume these as well.

In {{TarArchiveInputStream.getRecord()}}, I suggest to replace

{code}
  } else if (buffer.isEOFRecord(headerBuf)) {
    hasHitEOF = true;
  }
{code}

with

{code}
  } else if (buffer.isEOFRecord(headerBuf)) {
    while (buffer.readRecord() != null) { // Consume any lingering records
      ;
    }
    hasHitEOF = true;
  }
{code}

This fixes the test. It doesn't seem to break any other tests either, although I did not run
all of them because they take a long time and I didn't have the patience.
                
      was (Author: peterdm):
    I think the root cause is that TarArchiveInputStream stops reading the input stream once
it hits the first EOF record. Any lingering records after that are left unconsumed. It would
be best to consume these as well.

In {{TarArchiveInputStream.getRecord()}}, I suggest to replace

{code}
  ...
  } else if (buffer.isEOFRecord(headerBuf)) {
    hasHitEOF = true;
  }
  ...
{code}

with

{code}
  } else if (buffer.isEOFRecord(headerBuf)) {
    while (buffer.readRecord() != null) { // Consume any lingering records
      ;
    }
    hasHitEOF = true;
  }
{code}

This fixes the test. It doesn't seem to break any other tests either, although I did not run
all of them because they take a long time and I didn't have the patience.
                  
> TarArchiveOutputStream sometimes writes garbage beyond the end of the archive
> -----------------------------------------------------------------------------
>
>                 Key: COMPRESS-206
>                 URL: https://issues.apache.org/jira/browse/COMPRESS-206
>             Project: Commons Compress
>          Issue Type: Bug
>          Components: Compressors
>    Affects Versions: 1.0, 1.4.1
>         Environment: Linux x86
>            Reporter: Peter De Maeyer
>         Attachments: GarbageBeyondEndTest.java
>
>
> For some combinations of file lengths, the archive created by TarArchiveOutputStream
writes garbage beyond the end of the TAR stream. TarArchiveInputStream can still read the
stream without problems, but it does not read beyond the garbage. This is problematic for
my use case because I write a checksum _after_ the TAR content. If I then try to read the
checksum back, I read garbage instead.
> Functional impact:
> * TarArchiveInputStream is asymmetrical with respect to TarArchiveOutputStream, in the
sense that TarArchiveInputStream does not read everything that was written by TarArchiveOutputStream.
> * The content is unnecessarily large. The garbage is totally unnecessarily large: ~10K
overhead compared to Linux command-line tar.
> This symptom is remarkably similar to #COMPRESS-81, which is supposedly fixed since 1.1.
Except for the fact that this issue still exists... I've tested this with 1.0 and 1.4.1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message