hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lavkesh Lahngir <lavk...@linux.com>
Subject Tuning thread counts in yarn production cluster.
Date Wed, 04 Feb 2015 03:51:53 GMT
Needed suggestions on how to arrive at thread counts for Yarn cluster.

Hadoop version: 2.6.0
Number of nodes in cluster : 250 .
Node configuration:  24cores, 64 GB RAM and 6 * 1 TB (7200 RPM)
Expected number of jobs at peak time: 1500
Maximum number of containers required by a job: 80k-100k

We are trying to tune various thread counts.

yarn.resourcemanager.client.thread-count
yarn.resourcemanager.scheduler.client.thread-count
yarn.resourcemanager.resource-tracker.client.thread-count
yarn.nodemanager.container-manager.thread-count
yarn.app.mapreduce.am.job.task.listener.thread-count

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message