flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mxm <...@git.apache.org>
Subject [GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...
Date Thu, 09 Jul 2015 13:46:44 GMT
GitHub user mxm opened a pull request:

    https://github.com/apache/flink/pull/896

    [FLINK-2292][FLINK-1573] add live per-task accumulators

    This refactors the accumulators to accumulate per task execution. The
    accumulators are reported from the task managers periodically to the job
    manager via the Heartbeat message. If the execution contains chained
    tasks, the accumulators are chained as well. The final accumulator
    results are reported via the UpdateTaskExecutionState message.
    
    The accumulators are now saved in the Execution within the
    ExecutionGraph. This makes the AccumulatorManager obsolete. It has been
    removed for now. In the future, we might introduce some caching for the
    web frontend visualization.
    
    Two types of accumulators are available:
    
    - external (user-defined via the RuntimeContext)
    - internal (flink metrics defined in the invocables)
    
    The internal (built-in) metrics are targeted at users who want to
    monitor their programs, e.g. through the job manager's web frontend.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mxm/flink live-accumulators

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/896.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #896
    
----
commit 7cec1236f087e72b40022bf02a6dbb12d74acbac
Author: Maximilian Michels <mxm@apache.org>
Date:   2015-07-08T07:23:42Z

    [FLINK-2292][FLINK-1573] add live per-task accumulators
    
    This refactors the accumulators to accumulate per task execution. The
    accumulators are reported from the task managers periodically to the job
    manager via the Heartbeat message. If the execution contains chained
    tasks, the accumulators are chained as well. The final accumulator
    results are reported via the UpdateTaskExecutionState message.
    
    The accumulators are now saved in the Execution within the
    ExecutionGraph. This makes the AccumulatorManager obsolete. It has been
    removed for now. In the future, we might introduce some caching for the
    web frontend visualization.
    
    Two types of accumulators are available:
    
    - external (user-defined via the RuntimeContext)
    - internal (flink metrics defined in the invocables)
    
    The internal (built-in) metrics are targeted at users who want to
    monitor their programs, e.g. through the job manager's web frontend.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message