Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9C8BF200CBD for ; Thu, 6 Jul 2017 19:00:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 996F8166E8C; Thu, 6 Jul 2017 17:00:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DD824166E8A for ; Thu, 6 Jul 2017 19:00:03 +0200 (CEST) Received: (qmail 76238 invoked by uid 500); 6 Jul 2017 17:00:03 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 76227 invoked by uid 99); 6 Jul 2017 17:00:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Jul 2017 17:00:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id ABBD1C00B6 for ; Thu, 6 Jul 2017 17:00:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id yEEhzlhLpwzl for ; Thu, 6 Jul 2017 17:00:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 47FB05FC3D for ; Thu, 6 Jul 2017 17:00:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 8534EE08C6 for ; Thu, 6 Jul 2017 17:00:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3CCAB24670 for ; Thu, 6 Jul 2017 17:00:00 +0000 (UTC) Date: Thu, 6 Jul 2017 17:00:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-2534) KafkaIO should allow gaps in message offsets MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 06 Jul 2017 17:00:04 -0000 [ https://issues.apache.org/jira/browse/BEAM-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076898#comment-16076898 ] ASF GitHub Bot commented on BEAM-2534: -------------------------------------- Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3461 > KafkaIO should allow gaps in message offsets > -------------------------------------------- > > Key: BEAM-2534 > URL: https://issues.apache.org/jira/browse/BEAM-2534 > Project: Beam > Issue Type: Bug > Components: sdk-java-extensions > Affects Versions: 2.0.0 > Reporter: Raghu Angadi > Assignee: Raghu Angadi > Priority: Minor > Fix For: 2.1.0 > > > KafkaIO reader logs a warning when it notices gaps in offsets for messages. While such gaps are not expected for normal Kafka topics, there could be gaps when log compaction is enabled (which deletes older messages for a key). > This warning log is not very useful. Also we should take such gaps while estimating backlog. -- This message was sent by Atlassian JIRA (v6.4.14#64029)