hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gautam Singaraju <gautam.singar...@gmail.com>
Subject Re: RE: set how much CPU to be utilised by a MapReduce job
Date Tue, 26 Jan 2010 05:03:23 GMT
One thing that you might want to consider is to increase the
replication factor. This might take a lot of disk space, but also
might increase the performance.

You might also want to check out: Sun Grid Engine Hadoop Integration
http://blogs.sun.com/templedf/entry/beta_testing_the_sun_grid
---
Gautam



On Mon, Jan 25, 2010 at 11:38 AM, Todd Lipcon <todd@cloudera.com> wrote:
> If you can require a recent kernel, you could use cgroups:
>
> http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-and-projects.html
>
> No one has integrated this with hadoop yet as it's still pretty new, and
> Hadoop clusters are meant to be run on unshared hardware.
>
> -Todd
>
> On Mon, Jan 25, 2010 at 1:13 AM, Ryan Rawson <ryanobjc@gmail.com> wrote:
>
>> The only thing you could do is to have the tasktracker nice the child when
>> its exceeding its reservation. Aside from that, its hard to limit without
>> killing a process.
>>
>> On Jan 25, 2010 12:23 AM, "Naveen Kumar Prasad" <naveenkumarp@huawei.com>
>> wrote:
>>
>>
>> This functionality may not be readily available with Hadoop.
>>
>> But it would be appreciable if anyone can help me in understanding how to
>> go
>> about developing this feature.
>>
>> Regards, Naveen Kumar HUAWEI TECHNOLOGIES CO.,LTD. huawei_logo
>>
>> Address: Huawei Industrial Base Bantian Longgang Shenzhen 518129, P.R.China
>> www.huawei.com ---------...
>>
>> -----Original Message----- From: Allen Wittenauer [mailto:
>> awittenauer@linkedin.com] Sent: Monday, J...
>>
>

Mime
View raw message