Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of kwiley@keithwiley.com
 designates 69.56.216.30 as permitted sender)
From: Keith Wiley <kwiley@keithwiley.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Subject: Exec hadoop from Java, reuse JVM (client-side)?
Date: Wed, 1 Aug 2012 13:33:47 -0700
Message-Id: <3587332D-18B0-4E4D-8C62-010B7A276D4B@keithwiley.com>
To: common-user@hadoop.apache.org
Mime-Version: 1.0 (Apple Message framework v1084)

I know there is a lot of discussion about JVM reuse in Hadoop, but that =
usually refers to mappers running on the cluste nodesr.  I have a much =
different question.  I am running a Java program which at one point =
execs hadoop and that call sometimes fails in the fashion shown below.  =
Thus, this issue occurs entirely within the client machine (of course, I =
am currently running in pseudo-distributed mode which convolutes that =
point somewhat).  In other words, I successfully ran a Java program, but =
it failed to subsequently run *another* Java program (hadoop).  My =
interpretation of the hadoop startup scripts (the hadoop command itself =
for example) is that they run a second JVM in my scenario, and that they =
fail to allocate enough memory.

Is there any way to run hadoop from within a JVM such that it reuses the =
local JVM?

EXCEPTION: java.io.IOException: Cannot run program "hadoop": =
java.io.IOException: error=3D12, Cannot allocate memory
    java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    java.lang.Runtime.exec(Runtime.java:593)
    java.lang.Runtime.exec(Runtime.java:466)
    com.util.Shell.run(Shell.java:44)
    com.exe.Foo.bar(Foo.java:107)
    com.exe.Foo.run(Foo.java:205)
    com.exe.Foo.main(Foo.java:227)
Exception in thread "main" java.io.IOException: Cannot run program =
"hadoop": java.io.IOException: error=3D12, Cannot allocate memory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
	at java.lang.Runtime.exec(Runtime.java:593)
	at java.lang.Runtime.exec(Runtime.java:466)
	at com.util.Shell.run(Shell.java:44)
	at com.exe.Foo.bar(Foo.java:107)
	at com.exe.Foo.run(Foo.java:205)
	at com.exe.Foo.main(Foo.java:227)
Caused by: java.io.IOException: java.io.IOException: error=3D12, Cannot =
allocate memory
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
	at java.lang.ProcessImpl.start(ProcessImpl.java:65)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
	... 6 more

=
__________________________________________________________________________=
______
Keith Wiley     kwiley@keithwiley.com     keithwiley.com    =
music.keithwiley.com

"You can scratch an itch, but you can't itch a scratch. Furthermore, an =
itch can
itch but a scratch can't scratch. Finally, a scratch can itch, but an =
itch can't
scratch. All together this implies: He scratched the itch from the =
scratch that
itched but would never itch the scratch from the itch that scratched."
                                           --  Keith Wiley
=
__________________________________________________________________________=
______