beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Work logged] (BEAM-4374) Update existing metrics in the FN API to use new Metric Schema
Date Mon, 24 Sep 2018 23:40:00 GMT


ASF GitHub Bot logged work on BEAM-4374:

                Author: ASF GitHub Bot
            Created on: 24/Sep/18 23:39
            Start Date: 24/Sep/18 23:39
    Worklog Time Spent: 10m 
      Work Description: ajamato commented on issue #6205: [BEAM-4374] Implementing a subset
of the new metrics framework in python.
   > @ajamato A more general question for my knowledge: as the communication between runner
harness and SDK harness is done on a bundle basis. When the runner harness sends data to the
sdk harness to execute a transform that contains metrics, does it:
   > 1. send metrics values (for the ones defined in the transform) alongside with data
and receive an updated value of the metrics when the bundle is finished processing?
   > 2. or does it send only the data and the sdk harness responds with a diff value of
the metrics so that the runner can update them in its side?
   > My bet is option 2. But can you confirm?
   Option 2, the metric is calculated for the bundle. That is, we send a single metric update
for the bundle. The design expects that the upstream systems will handle that aggregation.
It may sum all the metrics up together, outside of the SDK. The runner should forward these
metrics to a metric aggregation system across to aggregate the metrics to a final value across
all the workers.

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

Issue Time Tracking

    Worklog Id:     (was: 147383)
    Time Spent: 6.5h  (was: 6h 20m)

> Update existing metrics in the FN API to use new Metric Schema
> --------------------------------------------------------------
>                 Key: BEAM-4374
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Alex Amato
>            Priority: Major
>          Time Spent: 6.5h
>  Remaining Estimate: 0h
> Update existing metrics to use the new proto and cataloging schema defined in:
> [_]
>  * Check in new protos
>  * Define catalog file for metrics
>  * Port existing metrics to use this new format, based on catalog names+metadata

This message was sent by Atlassian JIRA

View raw message