flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wangzhijiang999" <wangzhijiang...@aliyun.com>
Subject thread model issue in TaskManager
Date Thu, 30 Jul 2015 02:02:31 GMT
As I know, flink uses thread model in TaskManager, that means one taskmanager process may run
many different operator threads,and these threads will compete the memory of the process.
I know that flink has memoryManage component in each taskManager, and it will control the
localBufferPool of InputGate, ResultPartition for each task,but if UDF consume much memory,
it will use jvm heap memory, so it can not be controlled by flink. If I use flink as common
platform, some users will consume much memory in UDF, and it may influence other threads in
the process, especially for OOM.  I know that it has sharedslot or isolated slot properties
, but it just limit the task schedule in one taskmanager, can i schedule task in separate
taskmanger if i consume much memory and donot want to influence other tasks. Or are there
any suggestions for the issue of thread model. As I know spark is also thread model, but hadoop2
use process model.
Thank you for any suggestions in advance!
Mime
View raw message