flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yun Gao" <yungao...@aliyun.com.INVALID>
Subject Re: OutputFlusher线程创建太多
Date Thu, 19 Sep 2019 09:44:37 GMT
理论上来说每个ResultPartition会创建一个OutputFlush线程,这个作业一个TM上面有多少个Task?
每个Task有多少个ResultPartition?


------------------------------------------------------------------
From:李杰 <arjenlee1992@gmail.com>
Send Time:2019 Sep. 19 (Thu.) 10:02
To:user-zh <user-zh@flink.apache.org>
Subject:OutputFlusher线程创建太多

flink 1.7.0
on yarn模式
=========================
程序运行了大概一周,检测发现YarnTaskExecutorRunner,线程数达彪到了10000左右,正常来说应该不到400.
dump线程栈发现大量 的【接近1w】 的 TIMED_WAITING (sleeping) 状态的outputFluster
"OutputFlusher for xxxx -> yyyy" #1017018 daemon prio=5 os_prio=0
tid=0x0000000001bb6000 nid=0x894a sleeping[0x00002b0f6c42b000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.flink.streaming.runtime.io.StreamRecordWriter$OutputFlusher.run(StreamRecordWriter.java:161)

   Locked ownable synchronizers:
- None

大家碰有到过类似的情况吗?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message