kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ismael Juma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4935) Consider disabling record level CRC checks for message format V2
Date Wed, 05 Apr 2017 23:40:42 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958033#comment-15958033

Ismael Juma commented on KAFKA-4935:

Whatever we decide, we should remove the following code comment:

// TODO: The crc is useless for the new message format. Maybe we should let writeTo return
the written size?

> Consider disabling record level CRC checks for message format V2
> ----------------------------------------------------------------
>                 Key: KAFKA-4935
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4935
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: Apurva Mehta
>             Fix For:
> With the new message format proposed in KIP-98, the record level CRC has been moved to
the the batch header.
> Because we expose the record-level CRC in `RecordMetadata` and `ConsumerRecord`, we currently
compute it eagerly based on the key, value and timestamp even though these methods are rarely
used. Ideally, we'd deprecate the relevant methods in `RecordMetadata` and `ConsumerRecord`
while making the CRC computation lazy. This seems pretty hard to achieve in the Producer without
increasing memory retention, but it may be possible to do in the Consumer.
> An alternative option is to return the batch CRC from the relevant methods.

This message was sent by Atlassian JIRA

View raw message