crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pooja Dhondge (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-630) KafkaRecordReader keeps retrying to poll data when the offset is reset to latest offset
Date Mon, 12 Dec 2016 15:39:59 GMT
Pooja Dhondge created CRUNCH-630:
------------------------------------

             Summary: KafkaRecordReader keeps retrying to poll data when the offset is reset
to latest offset
                 Key: CRUNCH-630
                 URL: https://issues.apache.org/jira/browse/CRUNCH-630
             Project: Crunch
          Issue Type: Bug
            Reporter: Pooja Dhondge


We recently saw this behavior where, if the offset it is trying to read from doesn't exist
on Kafka due to retention policy, the offset gets reset to latest(default) and the KafkaRecordReader
keeps retrying beyond KAFKA_EMPTY_RETRY_ATTEMPTS_KEY

{noformat}
...crunch.kafka.inputformat.KafkaRecordReader: No records retrieved but pending offsets to
consume therefore polling again. Attempt 17/10
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message