giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ramani, Arun" <>
Subject Re: Giraph and Fair Scheduler
Date Tue, 07 May 2013 21:10:44 GMT
Hi Claudio,

Which timeout are you referring?

Arun Ramani

From: Claudio Martella <<>>
Reply-To: "<>" <<>>
Date: Tuesday, May 7, 2013 5:23 AM
To: "<>" <<>>
Subject: Re: Giraph and Fair Scheduler

it's probably an obvious question, but have you tried increasing the timeout?

On Tue, May 7, 2013 at 3:37 AM, Ramani, Arun <<>>

I am running Fair scheduler with many applications in hadoop stack in my cluster (like pig,
hive, hbase etc). I have dedicated a pool for Giraph and want to run giraph along with those
other applications. I have configured pre-emption and and set the "minsharepreemptiontimeout=5"
(sec – for the jobs submitted to this pool to wait to get the min share).

I am trying to run giraph in this mode. I see that jobs from other pools are getting pre-empted
to give the giraph job's pool its configured min share but my job fails with "Unable to create
native thread" error. This same job passes if the slots are available immediately without
having to wait for the tasks from other queues to be pre-empted. I also tried to tweak the
"giraph.minPercentResponded=50.0f". My Giraph job still fails. Please help in this scenario.

Basically, I wanted to know how to configure giraph to wait for a threshold for the slots
to be available for it through pre-emption.

Arun Ramani

   Claudio Martella<>

View raw message