flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KOSTIANTYN Kudriavtsev <kudryavtsev.konstan...@gmail.com>
Subject Debug OutOfMemory
Date Thu, 08 Oct 2015 15:58:16 GMT
Hi guys,

I'm running FLink on EMR with 2 m3.xlarge (each 16 GB RAM) and trying to
process 3.8 GB CSV data from S3. I'm surprised the fact that Flink failed
with OutOfMemory: Java Heap space

I tried to find the reason:
1) to identify TaskManager with a command ps aux | grep TaskManager
2) then build Heap histo:
$ jmap -histo:live 19648 | head -n23
 num     #instances         #bytes  class name
----------------------------------------------
   1:        131018     3763501304  [B
   2:         61022        7820352  <methodKlass>
   3:         61022        7688456  <constMethodKlass>
   4:          4971        5454408  <constantPoolKlass>
   5:          4966        4582232  <instanceKlassKlass>
   6:          4169        3003104  <constantPoolCacheKlass>
   7:         15696        1447168  [C
   8:          1291         638824  [Ljava.lang.Object;
   9:          5318         506000  java.lang.Class


Do you have any ideas what can be the reason and how it can be fixed?
Is Flink uses out-of-heap memory?


Thank you,
Konstantin Kudryavtsev

Mime
View raw message