flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-10252) Handle oversized metric messges
Date Tue, 16 Oct 2018 10:38:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651466#comment-16651466
] 

ASF GitHub Bot commented on FLINK-10252:
----------------------------------------

zentol commented on a change in pull request #6850: [FLINK-10252] Handle oversized metric
messges
URL: https://github.com/apache/flink/pull/6850#discussion_r225484559
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/dump/MetricQueryService.java
 ##########
 @@ -109,7 +122,17 @@ public void onReceive(Object message) {
 				}
 			} else if (message instanceof CreateDump) {
 				MetricDumpSerialization.MetricSerializationResult dump = serializer.serialize(counters,
gauges, histograms, meters);
-				getSender().tell(dump, getSelf());
+
+				int realMsgSize = dump.serializedMetrics.length;
+
+				if (realMsgSize > maximumFramesize) {
+					String overSizeErrorMsg = "The metric dump message size : " + realMsgSize
+						+ " exceeds the maximum akka framesize : " + maximumFramesize + ".";
+					LOG.error(overSizeErrorMsg);
+					getSender().tell(new Status.Failure(new IOException(overSizeErrorMsg)), getSelf());
 
 Review comment:
   alright, this isn't so easy to implement. I would suggest as a simple counter-measure to
retry the serialization without histograms (including unregistering all histograms and rejecting
future registrations), and if that fails as well to fail with an exception. Making this more
robust can be handled in a follow-up.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Handle oversized metric messges
> -------------------------------
>
>                 Key: FLINK-10252
>                 URL: https://issues.apache.org/jira/browse/FLINK-10252
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Metrics
>    Affects Versions: 1.5.3, 1.6.0, 1.7.0
>            Reporter: Till Rohrmann
>            Assignee: vinoyang
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> Since the {{MetricQueryService}} is implemented as an Akka actor, it can only send messages
of a smaller size then the current {{akka.framesize}}. We should check similarly to FLINK-10251
whether the payload exceeds the maximum framesize and fail fast if it is true.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message