hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "TooManyOpenFiles" by SteveLoughran
Date Fri, 16 Sep 2011 08:17:30 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "TooManyOpenFiles" page has been changed by SteveLoughran:

document the too  many open files problem and the two likely causes, quotas and limits on

New page:
= Too Many Open Files =

You can see this on Linux machines in client-side applications, server code or even in test

It is caused by per-process limits on the number of files that a single user/process can have
open, which was introduced in [[http://lkml.indiana.edu/hypermail/linux/kernel/0812.0/01183.html|2.6.27].
The default value, 128, was chosen because "that should be enough". 

In Hadoop, it isn't. To fix this log in/su/ssh as root and edit {{{/etc/sysctl.conf}}}

add the line
fs.epoll.max_user_instances = 2048

Then reboot. Different numbers may be chosen. 

There is an immediate shortcut: {{{echo 2048 > /proc/sys/fs/epoll/max_user_instances}}}
. This setting will be lost on the next reboot, but is handy for trying out different values.

== limits.conf ==

Another limit on the number of files open may be the file {{{/etc/security/limits.conf}}}

It has a setting on the number of files a user or group may have, {{{nofile}}}. 

To set this, as root edit {{{/etc/security/limits.conf}}}

and add a line such as 
*               soft    nofile            2048
Then restart.

To see the current/default limits, run the command {{{ulimit -a}}}. This should print something
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

You can dynamically up the limits until the next reboot with the same command. Specifically
ulimit -n 8192

The updated value can then be printed 
# ulimit -n

View raw message