hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sunww <spe...@outlook.com>
Subject hive client OutOfMemoryError
Date Sun, 28 Sep 2014 07:47:27 GMT
Hi:   I'm using hive0.11, When I run a sql with 2 union all, hive client throw exception:Exception
in thread "main" java.lang.OutOfMemoryError: Java
heap space 

at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:232) 

at java.lang.StringCoding.encode(StringCoding.java:272) 

at java.lang.String.getBytes(String.java:946) 

at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver$CommonJoinTaskDispatcher.processCurrentTask(CommonJoinResolver.java:589)


at
org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver$CommonJoinTaskDispatcher.dispatch(CommonJoinResolver.java:743)


at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)


at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)


at
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)


at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:112)


at
org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:79)


at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genMapRedTasks(SemanticAnalyzer.java:8399)


at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8741)


at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:278)


at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:433) 

at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) 

at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902) 

at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) 

at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) 

at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) 

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) 

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) 

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
at java.lang.reflect.Method.invoke(Method.java:597) 

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

I use "export HADOOP_CLIENT_OPTS=-Xmx1024m" to increase hive client heapsize, then the sql
is ok.I saw CommonJoinResolver  line 589:InputStream in = new ByteArrayInputStream(xml.getBytes("UTF-8"));
hive client need so many heapsize, is this normalAnd where Can I find this large xml

Thanks 		 	   		  
Mime
View raw message