kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Coon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-4224) IndexOutOfBounds in RecordsIterator causes infinite loop in NetworkClient
Date Tue, 27 Sep 2016 12:22:20 GMT
Michael Coon created KAFKA-4224:

             Summary: IndexOutOfBounds in RecordsIterator causes infinite loop in NetworkClient
                 Key: KAFKA-4224
                 URL: https://issues.apache.org/jira/browse/KAFKA-4224
             Project: Kafka
          Issue Type: Bug
          Components: consumer
    Affects Versions:
            Reporter: Michael Coon

For whatever reason, I seem to have a corrupted message that is returned from a broker that
puts the consumer into an infinite loop. The org.apache.kafka.client.consumer.internals.Fetcher
(line 590) is getting the next record from the RecordsIterator or MemoryRecords but when it
attempts to decode the record, it throws "IndexOutOfBounds" exception. Unfortunately, that
exception is merely logged and the Fetcher goes on to get the next message. But the exception
apparently does not move the underlying buffer read forward in such a way that it would actually
go and get the next record. The result: it keeps trying to read the corrupted record but can't
make progress. 

I offer two potential solutions: 

1) throw the exception up to me and let me figure out whether I want to skip forward in offsets
2) Make sure the underlying RecordsIterator actually moves forward on exceptions so that progress
can be made when corrupted messages are found.

This message was sent by Atlassian JIRA

View raw message