accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: Question about configuring the linux niceness of tablet servers?
Date Tue, 18 Aug 2015 22:29:50 GMT
So it sounds like what's going to have the biggest effect is making sure
the problematic mapreduce tasks run at a lower priority. I'd definitely be
interested in your results of you experiment with this.

On Tue, Aug 18, 2015, 17:33 Jeff Kubina <jeff.kubina@gmail.com> wrote:

> > I haven't heard of anybody setting the niceness of the Accumulo processes
>> > before. Are you experiencing a lot of CPU contention on your nodes, such
>> > that you need to prioritize processes?
>>
>
> Yes, we have had mapreduce jobs "lock out" the tserver so long that the
> master removes them from its pool for not responding. But also when doing a
> query from another service while the mapreduce jobs are maxed out I want to
> make sure the tserver takes priority.
>
>
>> > However, if you do this, I think you actually want HDFS to -2, Accumulo
>> to -1,
>> > and Mapreduce tasks 0 (you'll have to edit the system security limits
>> to allow
>> > niceness below 0 for non-root users).
>> > Otherwise, you're lowering the priority, not increasing it (and
>> lowering it
>> > below other, normal user processes, since the default niceness is 0).
>>
>
> Agreed. That is what I wanted to do originally but it requires much more
> reconfiguration. Ideally all OS root processes currently at 0 get reniced
> to -2, HDFS to -2, Accumulo to -2, and mapreduce to -1 and all other user
> processes stay at 0. Given the amount of threading a datanode does by
> default it could lockout a tserver if its priority was lower, so I think
> the HDFS and Accumulo should have equal priority.
>
>

Mime
View raw message