kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Gustafson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4224) IndexOutOfBounds in RecordsIterator causes infinite loop in NetworkClient
Date Tue, 27 Sep 2016 18:52:20 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527086#comment-15527086

Jason Gustafson commented on KAFKA-4224:

I understand that we catch exceptions raised from response handlers in NetworkClient. Whether
or not this is ideal, it should not affect the ability to propagate errors to the user. In
other words, if there is an exception which is being inappropriately caught at that layer,
it is a bug which should be fixed. But to determine if that is the case, we need to know which
exception is being caught. If this is happening, there should be an exception trace, since
we do log the exception.

> IndexOutOfBounds in RecordsIterator causes infinite loop in NetworkClient
> -------------------------------------------------------------------------
>                 Key: KAFKA-4224
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4224
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions:
>            Reporter: Michael Coon
> For whatever reason, I seem to have a corrupted message that is returned from a broker
that puts the consumer into an infinite loop. The org.apache.kafka.client.consumer.internals.Fetcher
(line 590) is getting the next record from the RecordsIterator or MemoryRecords but when it
attempts to decode the record, it throws "IndexOutOfBounds" exception. Unfortunately, that
exception is merely logged and the Fetcher goes on to get the next message. But the exception
apparently does not move the underlying buffer read forward in such a way that it would actually
go and get the next record. The result: it keeps trying to read the corrupted record but can't
make progress. 
> I offer two potential solutions: 
> 1) throw the exception up to me and let me figure out whether I want to skip forward
in offsets
> 2) Make sure the underlying RecordsIterator actually moves forward on exceptions so that
progress can be made when corrupted messages are found.

This message was sent by Atlassian JIRA

View raw message