kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Koshy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-75) Corp replica does not shutdown on IO error
Date Tue, 26 Jul 2011 18:29:09 GMT
Corp replica does not shutdown on IO error

                 Key: KAFKA-75
                 URL: https://issues.apache.org/jira/browse/KAFKA-75
             Project: Kafka
          Issue Type: Improvement
            Reporter: Joel Koshy

The embedded consumer in the corp replica uses the low-level Log api to create the replica.
The append operation may fail and result in a corrupt log file, due to an IO error which is
currently caught and ignored.

The proposed fix is to switch to using the high-level producer API to create the replica.
Not only would this avoid the above issue, but it would also fit better with the current design
of the replication enhancement for kafka (http://linkedin.jira.com/browse/KAFKA-23), since
the low-level Log api is not replication-aware. Another advantage is that compression is exposed
at the producer API-level. One caveat in this approach would be the following: the async producer
drops events when its queue is full. This behavior is unsuitable for the embedded consumer,
so we can expose a configuration option in the producer to allow for (queue-level) blocking

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message