kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ismael Juma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4745) KafkaLZ4BlockOutputStream.java incorrectly finishes the last frame
Date Wed, 08 Feb 2017 23:18:42 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15858687#comment-15858687
] 

Ismael Juma commented on KAFKA-4745:
------------------------------------

Also `writeEndMark` sets finished=true as well.

> KafkaLZ4BlockOutputStream.java incorrectly finishes the last frame
> ------------------------------------------------------------------
>
>                 Key: KAFKA-4745
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4745
>             Project: Kafka
>          Issue Type: Bug
>          Components: compression
>    Affects Versions: 0.10.1.1
>            Reporter: Will Droste
>             Fix For: 0.10.1.1
>
>
> There is a scenario where by the delegated OutputStream does not call flush before close
there will be missing data in the stream. The reason for this is the stream is actually marked
close before it is actually flushed.
> The end mark is written before the flush, also the writeEndMark was finishing the stream
so its redundant in this context to mark it finished. In my fork the 'finished=true' was removed
from the 'writeEndMark' method.
> {code}
>     @Override
>     public void close() throws IOException {
>         if (!finished) {
>             writeEndMark();
>             flush();
>             finished = true;
>         }
>         if (out != null) {
>             out.close();
>             out = null;
>         }
>     }
> {code}
> should be
> {code}
>     @Override
>     public void close() throws IOException {
>         if (!finished) {
>             // finish any pending data
>             writeBlock();
>             // write out the end mark
>             writeEndMark();
>             // mark the stream as finished
>             finished = true;
>         }
>         if (out != null) {
>             out.close();
>             out = null;
>         }
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message