kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4745) KafkaLZ4BlockOutputStream.java incorrectly finishes the last frame
Date Thu, 09 Feb 2017 18:03:41 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859922#comment-15859922
] 

ASF GitHub Bot commented on KAFKA-4745:
---------------------------------------

GitHub user wdroste opened a pull request:

    https://github.com/apache/kafka/pull/2528

    KAFKA-4745 -Optimize close to remove unnecessary flush

    -Remove unnecessary 'flush', the underlying stream should handle it on close, also most
apps flush before close.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/wdroste/kafka trunk

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2528.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2528
    
----
commit d4a939e14879634be52976a92c55e0181b8addcb
Author: Will Droste <william.droste@arris.com>
Date:   2017-02-09T18:00:37Z

    KAFKA-4745 -Remove unnecessary 'flush', the underlying stream should handle it on close,
also most apps flush before close.

----


> KafkaLZ4BlockOutputStream.java incorrectly finishes the last frame
> ------------------------------------------------------------------
>
>                 Key: KAFKA-4745
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4745
>             Project: Kafka
>          Issue Type: Bug
>          Components: compression
>    Affects Versions: 0.10.1.1
>            Reporter: Will Droste
>             Fix For: 0.10.1.1
>
>
> There is a scenario where by the delegated OutputStream does not call flush before close
there will be missing data in the stream. The reason for this is the stream is actually marked
close before it is actually flushed.
> The end mark is written before the flush, also the writeEndMark was finishing the stream
so its redundant in this context to mark it finished. In my fork the 'finished=true' was removed
from the 'writeEndMark' method.
> {code}
>     @Override
>     public void close() throws IOException {
>         if (!finished) {
>             writeEndMark();
>             flush();
>             finished = true;
>         }
>         if (out != null) {
>             out.close();
>             out = null;
>         }
>     }
> {code}
> should be
> {code}
>     @Override
>     public void close() throws IOException {
>         if (!finished) {
>             // finish any pending data
>             writeBlock();
>             // write out the end mark
>             writeEndMark();
>             // mark the stream as finished
>             finished = true;
>         }
>         if (out != null) {
>             out.close();
>             out = null;
>         }
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message