hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "pavel kolodin" <pavelkolodinhad...@gmail.com>
Subject Works on laptop with 2GB, but cannnot allocate memory on VPS with 3.5 GB.
Date Sun, 13 Dec 2009 10:58:54 GMT

I am running hadoop-0.20.1.
conf/hadoop-env.sh have default setting, excluding one "JAVA_HOME".

Success with 2 such nodes:
1) laptop, pentium M760, 2GB RAM
2) VirtualBox running on this laptop with 350MB allowed "RAM"
(all - gentoo linux)

Failed with 2 such nodes:
1) VPS, 64bit, 3.5GB RAM
2) VPS, 64bit, 2.5GB RAM
(all - gentoo linux, hardware - 1 server, 16 Xeons, 16GB)

Many errors, all because of deficit of memory:
* "DrWho" (whoami available, but can't start because of memory deficit)
* "java.io.IOException: Cannot run program "chmod": java.io.IOException:  
error=12, Cannot allocate memory"
* and other...

So, it can allocate memory when running on laptop and can't allocate  
memory running on VPS with 3.5 GB.
I have write small C++ program, that do "char *p = new char [ 1024*1024  
];" many times (ok, ~3500 times, little less than 3.5GB), and i see that  
program can allocate about ~3.5GB successfully and then deallocate it, so  
memory system is OK )

Any ideas?
Thank you.

I played with -Xmx in HADOOP_*_OPTS in hadoop-env.sh - not successful.  
(strange effects described in this mailing list earlier, subject was:  
"Memory. -Xmx. error=12, chmod, JobTracker")

(sorry, previous sent from wrong email :) :) :)

View raw message