beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Etienne Chauchot (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-3310) Push metrics to a backend in an runner agnostic way
Date Wed, 06 Dec 2017 16:06:00 GMT
Etienne Chauchot created BEAM-3310:
--------------------------------------

             Summary: Push metrics to a backend in an runner agnostic way
                 Key: BEAM-3310
                 URL: https://issues.apache.org/jira/browse/BEAM-3310
             Project: Beam
          Issue Type: New Feature
          Components: sdk-java-core
            Reporter: Etienne Chauchot


The idea is to avoid relying on the runners to provide access to the metrics (either at the
end of the pipeline or while it runs) because they don't have all the same capabilities towards
metrics (e.g. spark runner configures sinks  like csv, graphite or in memory sinks using the
spark engine conf). The aim is to push metrics to a sink when they are updated. The target
is to push the metrics in the common sdk code so that no matter the chosen runner, a user
can get his metrics out of beam.
Here is the link to the discussion thread on the dev ML: https://lists.apache.org/thread.html/01a80d62f2df6b84bfa41f05e15fda900178f882877c294fed8be91e@%3Cdev.beam.apache.org%3E



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message