hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lavkesh Lahngir <lavk...@linux.com>
Subject Re: Tuning thread counts in yarn production cluster.
Date Wed, 04 Feb 2015 04:16:29 GMT
On Wed, Feb 4, 2015 at 9:28 AM, Allen Wittenauer <aw@altiscale.com> wrote:

>
> On Feb 3, 2015, at 7:51 PM, Lavkesh Lahngir <lavkesh@linux.com> wrote:
>
> > Expected number of jobs at peak time: 1500
> > Maximum number of containers required by a job: 80k-100k
>
>         If that is really per job, you might want to tune your jobs
> first.  That's extremely high.


These numbers are not per job. These are the extreme cases. Some analytics
queries with large date range can spawn these many mapper/reducers. On an
average total number of containers will be lesser.

>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message