Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 48AC4200CD0 for ; Tue, 11 Jul 2017 00:18:12 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 4703E1646D1; Mon, 10 Jul 2017 22:18:12 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8D7A91646B2 for ; Tue, 11 Jul 2017 00:18:11 +0200 (CEST) Received: (qmail 31577 invoked by uid 500); 10 Jul 2017 22:18:10 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 31568 invoked by uid 99); 10 Jul 2017 22:18:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Jul 2017 22:18:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 01C291945CE for ; Mon, 10 Jul 2017 22:18:10 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 3GZXwuVwa8zB for ; Mon, 10 Jul 2017 22:18:09 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id C6D6A62951 for ; Mon, 10 Jul 2017 19:55:00 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 5EEB9E0D71 for ; Mon, 10 Jul 2017 19:55:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 1267624695 for ; Mon, 10 Jul 2017 19:55:00 +0000 (UTC) Date: Mon, 10 Jul 2017 19:55:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-2534) KafkaIO should allow gaps in message offsets MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 10 Jul 2017 22:18:12 -0000 [ https://issues.apache.org/jira/browse/BEAM-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16080984#comment-16080984 ] ASF GitHub Bot commented on BEAM-2534: -------------------------------------- Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3514 > KafkaIO should allow gaps in message offsets > -------------------------------------------- > > Key: BEAM-2534 > URL: https://issues.apache.org/jira/browse/BEAM-2534 > Project: Beam > Issue Type: Bug > Components: sdk-java-extensions > Affects Versions: 2.0.0 > Reporter: Raghu Angadi > Assignee: Raghu Angadi > Priority: Minor > Fix For: 2.1.0 > > > KafkaIO reader logs a warning when it notices gaps in offsets for messages. While such gaps are not expected for normal Kafka topics, there could be gaps when log compaction is enabled (which deletes older messages for a key). > This warning log is not very useful. Also we should take such gaps while estimating backlog. -- This message was sent by Atlassian JIRA (v6.4.14#64029)