ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Custom thread pools for compute tasks
Date Sat, 11 Feb 2017 20:43:53 GMT
Igniters,

We have one well-known limitation: one distributed opeartion cannot be
executed from within another if they both end up in the same pool, because
it will lead to starvation and hang. Most notable example - running data
streamer from compute task.

Users with complex logic will have the same issue if they want to execute
one job from another. I think it could be resolved with introduction of
custom executors and routing computing jobs to them.

Config:
ExecutorConfiguration execCfg = new
ExecutorConfiguration().setName("myExecutor").setThreads(8);

Starting MR:
ignite.compute().withExecutor("myExecutor").execute(...);

Another improvement we can apply here is managed blockers, similar to
ForkJoinPool.ManagedBlocker [1]. If user is to execute distributed
operation, he can infrom us about it, and we will compensate blocked thread
with a new one.

Thoughts?

[1]
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.ManagedBlocker.html

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message