flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Hogan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-4452) TaskManager network buffer gauges
Date Tue, 23 Aug 2016 19:03:21 GMT

     [ https://issues.apache.org/jira/browse/FLINK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Greg Hogan updated FLINK-4452:
------------------------------
    Description: 
Add gauges for {{network.getNetworkBufferPool.getTotalNumberOfMemorySegments}} and {{network.getNetworkBufferPool.getNumberOfAvailableMemorySegments}}.

Providing insight into the number and proportion of used network buffers is vital and enlightening.
Jobs terminate when buffers are not available, but also the rule-of-thumb for "Configuring
the Network Buffers" from the documentation is way off. For example, running a sort on a single
TaskManager with 8 slots I am using 16,000+ buffers which is much greater than 8*8*4 = 256.

  was:
Add guages for {{network.getNetworkBufferPool.getTotalNumberOfMemorySegments}} and {{network.getNetworkBufferPool.getNumberOfAvailableMemorySegments}}.

Providing insight into the number and proportion of used network buffers is vital and enlightening.
Jobs terminate when buffers are not available, but also the rule-of-thumb for "Configuring
the Network Buffers" from the documentation is way off. For example, running a sort on a single
TaskManager with 8 slots I am using 16,000+ buffers which is much greater than 8*8*4 = 256.


> TaskManager network buffer gauges
> ---------------------------------
>
>                 Key: FLINK-4452
>                 URL: https://issues.apache.org/jira/browse/FLINK-4452
>             Project: Flink
>          Issue Type: New Feature
>          Components: Metrics
>    Affects Versions: 1.2.0
>            Reporter: Greg Hogan
>            Assignee: Greg Hogan
>            Priority: Minor
>
> Add gauges for {{network.getNetworkBufferPool.getTotalNumberOfMemorySegments}} and {{network.getNetworkBufferPool.getNumberOfAvailableMemorySegments}}.
> Providing insight into the number and proportion of used network buffers is vital and
enlightening. Jobs terminate when buffers are not available, but also the rule-of-thumb for
"Configuring the Network Buffers" from the documentation is way off. For example, running
a sort on a single TaskManager with 8 slots I am using 16,000+ buffers which is much greater
than 8*8*4 = 256.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message