hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bikash sharma <sharmabiks...@gmail.com>
Subject Re: conceptual question regarding slots
Date Thu, 03 Mar 2011 00:19:36 GMT
Thanks Greg!

On Wed, Mar 2, 2011 at 5:07 PM, Greg Roelofs <roelofs@yahoo-inc.com> wrote:

> > Could someone throw some light as to how intuitively fixed-type slots in
> > Hadoop have a negative impact of cluster utilization as mentioned in
> Arun's
> > blog?
>
> It's pretty simple:  in the course of time, a real cluster with a mix of
> job types and submission times will have periods of high map-slot usage
> (with reduce-slots available), and vice versa.  Being able to assign
> machine
> resources to whatever kind of task needs them at any given time means
> you're
> not wasting those resources.
>
> In short:  map- vs. reduce-slots is an artificial and unnecessary
> restriction.
>
> Greg
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message