hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4391) Add ability to start RS as root and call mlockall
Date Tue, 13 Sep 2011 23:16:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104074#comment-13104074
] 

Todd Lipcon commented on HBASE-4391:
------------------------------------

I've put an initial implementation of a JVMTI agent that accomplishes this here:

https://github.com/toddlipcon/mlockall_agent

It can be compiled using the "compile.sh" script in the repo, and then configured like so
in hbase-env.sh:
{code}
export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -agentpath:/home/todd/git/libmlockall_agent/libmlockall_agent.so=user=todd"
{code}

After which point you must run the regionserver as root:
{code}

todd@todd-w510:~/git/hbase$ ./bin/hbase regionserver 
mlockall_agent: Unable to boost memlock resource limit: Operation not permitted
mlockall_agent: (this may be because java was not run as root!)
Error occurred during initialization of VM
agent library failed to init: /home/todd/git/libmlockall_agent/libmlockall_agent.so
todd@todd-w510:~/git/hbase$ sudo ./bin/hbase regionserver 
mlockall_agent: Successfully locked memory and setuid to todd
{code}

Perhaps some other people can check this out, and if it looks generally useful we could either
build it along with HBase or at least point to it from docs?

> Add ability to start RS as root and call mlockall
> -------------------------------------------------
>
>                 Key: HBASE-4391
>                 URL: https://issues.apache.org/jira/browse/HBASE-4391
>             Project: HBase
>          Issue Type: New Feature
>          Components: regionserver
>    Affects Versions: 0.94.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.94.0
>
>
> A common issue we've seen in practice is that users oversubscribe their region servers
with too many MR tasks, etc. As soon as the machine starts swapping, the RS grinds to a halt,
loses ZK session, aborts, etc.
> This can be combatted by starting the RS as root, calling mlockall(), and then setuid
down to the hbase user. We should not require this, but we should provide it as an option.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message