giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ramani, Arun" <aram...@paypal.com>
Subject Re: Giraph and Fair Scheduler
Date Tue, 07 May 2013 21:26:43 GMT
Hi Avery,

I am setting "minsharepreemptiontimeout" to 5 sec and my Giraph job could not even wait for
5 secs to get its slots. Let me explain the scenario below:

Assume, Cluster capacity is 150
Queue A (min share –10 maps) - I submit a sleep job with 100 map tasks. Cluster is empty,
and hence the first job submitted to Queue A will take the entire 100 map tasks.
Queue B (Giraph pool with min share – 140 maps) - Now my job 1 is running with 100 tasks
occupied. I submit a giraph shortestpathfirst example job with 100 workers to Queue B. Queue
B has "minsharepreemptiontimeout" to 5 sec". So, it will first schedule 50 tasks since first
job only took 100 tasks and cluster's capacity is 150. Meanwhile, in 5 sec, 50 more tasks
would be preempted from Queue A and would be given to Giraph Job. I see this happening, however,
the job fails with "Unable to create native thread error"

Please let me know if "giraph.maxMasterSuperstepWaitMsecs" will help in this scenario.

Thanks so much
Arun Ramani

From: Avery Ching <aching@apache.org<mailto:aching@apache.org>>
Date: Tuesday, May 7, 2013 2:19 PM
To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>>
Cc: "Ramani, Arun(aramani)" <aramani@paypal.com<mailto:aramani@paypal.com>>
Subject: Re: Giraph and Fair Scheduler

Oh, I see.  You can change the timeout of how long the giraph job waits for tasks before giving
up.  Try setting giraph.maxMasterSuperstepWaitMsecs to a higher number.  The default is 10
minutes.

Avery

On 5/7/13 2:10 PM, Ramani, Arun wrote:
Hi Avery,

I am not preempting tasks out of the giraph pool. I have configured pre-emption so that any
job submitted to giraph pool will get its min share. Any suggestion on how to make this work?

Thanks so much in advance.

Arun Ramani

From: Avery Ching <aching@apache.org<mailto:aching@apache.org>>
Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>>
Date: Tuesday, May 7, 2013 7:25 AM
To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>>
Subject: Re: Giraph and Fair Scheduler

Can you disable the preemption for the giraph pool?  It's not great to preempt those tasks.

Avery

On 5/6/13 6:37 PM, Ramani, Arun wrote:
Hi,

I am running Fair scheduler with many applications in hadoop stack in my cluster (like pig,
hive, hbase etc). I have dedicated a pool for Giraph and want to run giraph along with those
other applications. I have configured pre-emption and and set the "minsharepreemptiontimeout=5"
(sec – for the jobs submitted to this pool to wait to get the min share).

I am trying to run giraph in this mode. I see that jobs from other pools are getting pre-empted
to give the giraph job's pool its configured min share but my job fails with "Unable to create
native thread" error. This same job passes if the slots are available immediately without
having to wait for the tasks from other queues to be pre-empted. I also tried to tweak the
"giraph.minPercentResponded=50.0f". My Giraph job still fails. Please help in this scenario.

Basically, I wanted to know how to configure giraph to wait for a threshold for the slots
to be available for it through pre-emption.

Thanks
Arun Ramani



Mime
View raw message