giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <>
Subject Re: Giraph and Fair Scheduler
Date Tue, 07 May 2013 14:25:43 GMT
Can you disable the preemption for the giraph pool?  It's not great to 
preempt those tasks.


On 5/6/13 6:37 PM, Ramani, Arun wrote:
> Hi,
> I am running Fair scheduler with many applications in hadoop stack in 
> my cluster (like pig, hive, hbase etc). I have dedicated a pool for 
> Giraph and want to run giraph along with those other applications. I 
> have configured pre-emption and and set the 
> "minsharepreemptiontimeout=5" (sec – for the jobs submitted to this 
> pool to wait to get the min share).
> I am trying to run giraph in this mode. I see that jobs from other 
> pools are getting pre-empted to give the giraph job's pool its 
> configured min share but my job fails with "Unable to create native 
> thread" error. This same job passes if the slots are available 
> immediately without having to wait for the tasks from other queues to 
> be pre-empted. I also tried to tweak the 
> "giraph.minPercentResponded=50.0f". My Giraph job still fails. Please 
> help in this scenario.
> Basically, I wanted to know how to configure giraph to wait for a 
> threshold for the slots to be available for it through pre-emption.
> Thanks
> Arun Ramani

View raw message