Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@crunch.apache.org
Date: Mon, 12 Dec 2016 15:39:59 +0000 (UTC)
From: "Pooja Dhondge (JIRA)" <jira@apache.org>
To: crunch-dev@incubator.apache.org
Message-ID: <JIRA.13027479.1481557194000.499101.1481557199739@Atlassian.JIRA>
In-Reply-To: <JIRA.13027479.1481557194000@Atlassian.JIRA>
References: <JIRA.13027479.1481557194000@Atlassian.JIRA> <JIRA.13027479.1481557194529@arcas>
Subject: [jira] [Created] (CRUNCH-630) KafkaRecordReader keeps retrying to
 poll data when the offset is reset to latest offset
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 12 Dec 2016 15:40:01 -0000

Pooja Dhondge created CRUNCH-630:
------------------------------------

             Summary: KafkaRecordReader keeps retrying to poll data when the offset is reset to latest offset
                 Key: CRUNCH-630
                 URL: https://issues.apache.org/jira/browse/CRUNCH-630
             Project: Crunch
          Issue Type: Bug
            Reporter: Pooja Dhondge


We recently saw this behavior where, if the offset it is trying to read from doesn't exist on Kafka due to retention policy, the offset gets reset to latest(default) and the KafkaRecordReader keeps retrying beyond KAFKA_EMPTY_RETRY_ATTEMPTS_KEY

{noformat}
...crunch.kafka.inputformat.KafkaRecordReader: No records retrieved but pending offsets to consume therefore polling again. Attempt 17/10
{noformat}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)