hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: RE: set how much CPU to be utilised by a MapReduce job
Date Mon, 25 Jan 2010 16:38:43 GMT
If you can require a recent kernel, you could use cgroups:

http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-and-projects.html

No one has integrated this with hadoop yet as it's still pretty new, and
Hadoop clusters are meant to be run on unshared hardware.

-Todd

On Mon, Jan 25, 2010 at 1:13 AM, Ryan Rawson <ryanobjc@gmail.com> wrote:

> The only thing you could do is to have the tasktracker nice the child when
> its exceeding its reservation. Aside from that, its hard to limit without
> killing a process.
>
> On Jan 25, 2010 12:23 AM, "Naveen Kumar Prasad" <naveenkumarp@huawei.com>
> wrote:
>
>
> This functionality may not be readily available with Hadoop.
>
> But it would be appreciable if anyone can help me in understanding how to
> go
> about developing this feature.
>
> Regards, Naveen Kumar HUAWEI TECHNOLOGIES CO.,LTD. huawei_logo
>
> Address: Huawei Industrial Base Bantian Longgang Shenzhen 518129, P.R.China
> www.huawei.com ---------...
>
> -----Original Message----- From: Allen Wittenauer [mailto:
> awittenauer@linkedin.com] Sent: Monday, J...
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message