hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michel Segel <michael_se...@hotmail.com>
Subject Re: Hadoop efficient resource isolation
Date Thu, 21 Feb 2013 16:37:05 GMT
Not sure what the question is... Have you looked at either the fair scheduler or better yet
capacity scheduler?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 21, 2013, at 5:16 AM, Dhanasekaran Anbalagan <bugcy013@gmail.com> wrote:

> Hi Guys,
> 
> It's possible isolation job submission for hadoop cluster, we currently running 48 machine
cluster. we  monitor Hadoop is not provides efficient resource isolation. In my case we ran
for tech and research pool, When tech job some memory leak will haven, It's occupy the hole
cluster.  Finally we figure out  issue with tech job. It's  screwed up hole hadoop cluster.
finally 10 data node  are dead.
> 
> Any prevention of job submission efficient way resource allocation. When something wrong
in   particular job, effect particular pool, Not effect others job. Any way to archive this
> 
> Please guide me guys.
> 
> My idea is, When tech user submit job means only apply job in for my case submit 24 machine.
other machine only for research user.
> 
> It's will prevent the memory leak problem. 
>  
> 
> -Dhanasekaran.
> Did I learn something today? If not, I wasted it.

Mime
View raw message