accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Medinets <david.medin...@gmail.com>
Subject Re: Does anyone know how to get a sqoop job into a specific fair scheduler pool?
Date Wed, 29 May 2013 14:59:35 GMT
Add the following to mapred-site.xml. Note that you can change the set of
queue names.

<property>
  <name>mapred.fairscheduler.poolnameproperty</name>
  <value>mapred.job.queue.name</value>
</property>
<property>
  <name>mapred.queue.names</name>
  <value>default,sqoop</value>
</property>

Define queues in the Fair Scheduler configuration file. They are
defined the same as
pools ... just use <queue> instead of <pool>.

Lastly, and this is the tricky part, use -Dmapred.job.queue.name *after* sqoop
import but *before* any other options.

sqoop import -Dmapred.job.queue.name=sqoop --user=ME --password=etc...

Unless I've forgotten something, that's all you need. It might be
important to know
that pools and queues seem to be separate. If you use the above
configuration, you
probably should always specify the job's queue name.

p.s. - is anyone else annoyed by the new gmail email editor?



On Wed, May 29, 2013 at 8:55 AM, Billie Rinaldi <billie.rinaldi@gmail.com>wrote:

> Nope.  The fair scheduler docs talk about how to configure which property
> determines the pool.  Have you looked at that?
> http://hadoop.apache.org/docs/stable/fair_scheduler.html
>
>
> On Wed, May 29, 2013 at 5:48 AM, David Medinets <david.medinets@gmail.com>wrote:
>
>> This is accumulo-related because the data is destined for Accumulo. :)
>>
>
>

Mime
View raw message