beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <>
Subject [jira] [Commented] (BEAM-1126) Expose UnboundedSource split backlog in number of events
Date Mon, 12 Dec 2016 17:07:58 GMT


Daniel Halperin commented on BEAM-1126:

This [thread on the dev list|]
has the additional context Davor is presumably asking for.

I think the confusion is between human-comprehensible and machine-comprehensible. Using {{bytes}}
as the measure of backlog was not written with PubSub in mind, it was written because bytes
is more directly related to overhead than events. Using bytes also allows for comparison between
sources of different types... so {{bytes}} is generally a pretty good signal for runners,
and better than {{events}}.

If the purpose of exposing {{events}} is purely for human visibility, this is probably indeed
better done using metric or aggregator reporting. [~bchambers] has been thinking most about
metrics recently, maybe he has additional thoughts?

> Expose UnboundedSource split backlog in number of events
> --------------------------------------------------------
>                 Key: BEAM-1126
>                 URL:
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-core
>            Reporter: Aviem Zur
>            Assignee: Daniel Halperin
>            Priority: Minor
> Today {{UnboundedSource}} exposes split backlog in bytes via {{getSplitBacklogBytes()}}
> There is value in exposing backlog in number of events as well, since this number can
be more human comprehensible than bytes. something like {{getSplitBacklogEvents()}} or {{getSplitBacklogCount()}}.

This message was sent by Atlassian JIRA

View raw message