hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chaitanya krishna" <chaitanyavv.ii...@gmail.com>
Subject Re: HELP: Namenode Startup Failed with an OutofMemoryError
Date Sun, 26 Oct 2008 04:09:18 GMT

 If the problem is due to the OS-level limit on the number of active
threads, then why is the error showing outofmemory exception? Is it an issue
of the heap size available for hadoop?Won't increasing heap size fix this

V.V.Chaitanya Krishna

On Fri, Oct 24, 2008 at 2:42 PM, Steve Loughran <stevel@apache.org> wrote:

> woody zhou wrote:
>> Hi everyone,
>> I have a problem about Hadoop startup.
>> I failed to startup the namenode and I got the following exception in the
>> namenode log file:
>  2008-10-23 21:54:51,232 ERROR org.apache.hadoop.dfs.NameNode:
>> java.lang.OutOfMemoryError: unable to create new native thread
>>        at java.lang.Thread.start0(Native Method)
>>        at java.lang.Thread.start(Thread.java:597)
>>        at org.apache.hadoop.ipc.Server.start(Server.java:991)
>>        at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:149)
>>        at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:193)
>>        at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:179)
>>        at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:830)
>>        at org.apache.hadoop.dfs.NameNode.main(NameNode.java:839)
>> How can I fix it? Does it mean that my machine doesn't have enough memory
>> for Hadoop startup?
>> Any help will be appreciated!
>> Thanks in advance.
>> - Woody
> Not memory; its an OS-level limit on the number of threads a process can
> have. Which is a mixture of physical and virtual memory allocation
> per-thread and any limits coded into the kernel.
> -search the web for the string "unable to create new native thread" and you
> will find more details and workarounds; include the OS you run on for some
> specific workarounds. For Hadoop, I'd consider throttling back the number of
> helpers
> 1. Have a look at the value of dfs.namenode.handler.count and set it to
> something lower
> 2. you can use kill -QUIT to get a dump of all threads in your process
> -this lets you see how many you have, and where they are.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message