flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ali, Kasif" <Kasif....@gs.com>
Subject flink akka OversizedPayloadException error
Date Wed, 15 Mar 2017 13:26:37 GMT
Hello,

We have added a serializer code which register all the schemas with executionEnvironment and
 is called before execute(). This is to make sure that all the avro schemas are pre-registered
and cached in executors before actual execution begin.  Now while submitting the job we are
getting below error.
executionEnvironment.addDefaultKryoSerializer(myKryo.class, largeSerializerObject);


Error :
13223 [flink-akka.actor.default-dispatcher-5] ERROR akka.remote.EndpointWriter - Transient
association error (association remains live)
akka.remote.OversizedPayloadException: Discarding oversized payload sent to Actor[akka.tcp://flink@d191303-012.dc.gs.com:54840/user/jobmanager#1440921956]:
max allowed size 10485760 bytes, actual size of encoded class org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage
was 45388747 bytes.

We followed the documentation to increase akka.framesize to 200mb while starting yarn cluster
but it still fails with above error.

akka.framesize: Maximum size of messages which are sent between the JobManager and the TaskManagers.
If Flink fails because messages exceed this limit, then you should increase it. The message
size requires a size-unit specifier (DEFAULT: 10485760b).


command to start yarn cluister:
./yarn-session.sh -n 15 -tm 24576 -jm 6144 -s 7 -Djobmanager.web.history=100 -Dakka.ask.timeout=10min
-Dtaskmanager.network.numberOfBuffers=8000 -Dakka.framesize=209715200b


We are using flink 1.2.0 version.


Can you suggest if this is correct config to increase akka frame size? Or Are we using correct
config to fix the issue?

Thanks,
Kasif


Mime
View raw message