Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id CBA98200B6F for ; Wed, 10 Aug 2016 08:22:24 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id CA45D160AAD; Wed, 10 Aug 2016 06:22:24 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1F541160AA5 for ; Wed, 10 Aug 2016 08:22:23 +0200 (CEST) Received: (qmail 18308 invoked by uid 500); 10 Aug 2016 06:22:23 -0000 Mailing-List: contact dev-help@apex.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@apex.apache.org Delivered-To: mailing list dev@apex.apache.org Received: (qmail 18297 invoked by uid 99); 10 Aug 2016 06:22:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Aug 2016 06:22:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D01F6C0F0F for ; Wed, 10 Aug 2016 06:22:22 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.652 X-Spam-Level: X-Spam-Status: No, score=-3.652 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.432] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id X5K6Ec1k5-ZE for ; Wed, 10 Aug 2016 06:22:22 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 53D535F3A0 for ; Wed, 10 Aug 2016 06:22:21 +0000 (UTC) Received: (qmail 18281 invoked by uid 99); 10 Aug 2016 06:22:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Aug 2016 06:22:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 71FEE2C02A1 for ; Wed, 10 Aug 2016 06:22:20 +0000 (UTC) Date: Wed, 10 Aug 2016 06:22:20 +0000 (UTC) From: "Siyuan Hua (JIRA)" To: dev@apex.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (APEXMALHAR-2158) Duplication of data emitted when the Kafka Input Operator(0.8 version) redeploys MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 10 Aug 2016 06:22:25 -0000 [ https://issues.apache.org/jira/browse/APEXMALHAR-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyuan Hua updated APEXMALHAR-2158: ----------------------------------- Fix Version/s: 3.5.0 > Duplication of data emitted when the Kafka Input Operator(0.8 version) redeploys > -------------------------------------------------------------------------------- > > Key: APEXMALHAR-2158 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2158 > Project: Apache Apex Malhar > Issue Type: Bug > Reporter: Chaitanya > Assignee: Chaitanya > Fix For: 3.5.0 > > > Kafka Input Operator failure recovery depends on offsetStats map. > "offsetStats" map is updated whenever the message is emitted from operator. > When the operator goes down and comes back, operator sets this map to the consumer and start reading from these offsets. Consumer reads the last emitted message again and operator emits it. > Solution: Increment the offsets before setting to the consumer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)