Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 28EE9200B13 for ; Wed, 15 Jun 2016 23:20:14 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 278D4160A4D; Wed, 15 Jun 2016 21:20:14 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 70B62160A19 for ; Wed, 15 Jun 2016 23:20:13 +0200 (CEST) Received: (qmail 17040 invoked by uid 500); 15 Jun 2016 21:20:12 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 17031 invoked by uid 99); 15 Jun 2016 21:20:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 21:20:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 4CDB2C0F43 for ; Wed, 15 Jun 2016 21:20:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id WcCWiAyX1JmQ for ; Wed, 15 Jun 2016 21:20:11 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id A43115F19D for ; Wed, 15 Jun 2016 21:20:10 +0000 (UTC) Received: (qmail 15575 invoked by uid 99); 15 Jun 2016 21:20:09 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 21:20:09 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 97C002C1F69 for ; Wed, 15 Jun 2016 21:20:09 +0000 (UTC) Date: Wed, 15 Jun 2016 21:20:09 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-347) Progress updates inaccurate for non-uniform keys in Bigtable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 15 Jun 2016 21:20:14 -0000 [ https://issues.apache.org/jira/browse/BEAM-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332596#comment-15332596 ] ASF GitHub Bot commented on BEAM-347: ------------------------------------- Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/440 > Progress updates inaccurate for non-uniform keys in Bigtable > ------------------------------------------------------------ > > Key: BEAM-347 > URL: https://issues.apache.org/jira/browse/BEAM-347 > Project: Beam > Issue Type: Improvement > Components: sdk-java-gcp > Reporter: Ian Zhou > Assignee: Daniel Halperin > Priority: Minor > > When reading from a Bigtable source with clustered keys, fraction consumed progress updates are inaccurate. For example, for a range spanning ['a', 'z'], a cluster of keys starting with the letter 'm' (e.g. 'me100,' ..., 'me999') will be recorded as ~50% complete upon reading the first key, and will remain at this percentage until the final key has been read. Instead, the start of the range should be changed to the first key read (e.g. new range ['me100', 'z']). The end of the range can be changed over time through dynamic work rebalancing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)