beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-347) Progress updates inaccurate for non-uniform keys in Bigtable
Date Wed, 15 Jun 2016 21:20:09 GMT

    [ https://issues.apache.org/jira/browse/BEAM-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332596#comment-15332596
] 

ASF GitHub Bot commented on BEAM-347:
-------------------------------------

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-beam/pull/440


> Progress updates inaccurate for non-uniform keys in Bigtable
> ------------------------------------------------------------
>
>                 Key: BEAM-347
>                 URL: https://issues.apache.org/jira/browse/BEAM-347
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-gcp
>            Reporter: Ian Zhou
>            Assignee: Daniel Halperin
>            Priority: Minor
>
> When reading from a Bigtable source with clustered keys, fraction consumed progress updates
are inaccurate. For example, for a range spanning ['a', 'z'], a cluster of keys starting with
the letter 'm' (e.g. 'me100,' ..., 'me999') will be recorded as ~50% complete upon reading
the first key, and will remain at this percentage until the final key has been read. Instead,
the start of the range should be changed to the first key read (e.g. new range ['me100', 'z']).
The end of the range can be changed over time through dynamic work rebalancing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message