Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of shujamughal@gmail.com
 designates 74.125.83.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=Dc2bUOWZI0shj+vsX2BpJm17qx3spFwguH0oqhjBS20408b882xQBe7cVLqhx9TYVF
         jz6X/0GhILveBSefYQWdzOXXm+y7NO3Jt+YFg6CLiyiootFRwnsEsiM6oDcTCOB93qUS
         nzxHOqNb5WcBcAxZi2iCp52Tvp5RFsEyQGoQA=
MIME-Version: 1.0
Date: Fri, 9 Jul 2010 22:14:55 +0500
Message-ID: <AANLkTiljuf4VYz5y627W589rKxHz-jllhsFy58stkF7K@mail.gmail.com>
Subject: java.lang.OutOfMemoryError: Java heap space
From: Shuja Rehman <shujamughal@gmail.com>
To: hive-user@hadoop.apache.org, general@hadoop.apache.org,
	mapreduce-user@hadoop.apache.org, mapreduce-issues@hadoop.apache.org
Content-Type: multipart/alternative; boundary=000e0cd28c90b1a0d2048af78a6a

--000e0cd28c90b1a0d2048af78a6a
Content-Type: text/plain; charset=ISO-8859-1

Hi All

I am facing a hard problem. I am running a map reduce job using streaming
but it fails and it gives the following error.

Caught: java.lang.OutOfMemoryError: Java heap space
	at Nodemapper5.parseXML(Nodemapper5.groovy:25)
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess
failed with code 1
	at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362)
	at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:572)
	at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:136)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
	at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)


I have increased the heap size in hadoop-env.sh and make it 2000M. Also I
tell the job manually by following line.

-D mapred.child.java.opts=-Xmx2000M \

but it still gives the error. The same job runs fine if i run on shell using
1024M heap size like

cat file.xml | /root/Nodemapper5.groovy


Any clue?????????

Thanks in advance.


-- 
Regards
Shuja-ur-Rehman Baig
_________________________________
MS CS - School of Science and Engineering
Lahore University of Management Sciences (LUMS)
Sector U, DHA, Lahore, 54792, Pakistan
Cell: +92 3214207445

--000e0cd28c90b1a0d2048af78a6a--