hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dima machlin (JIRA)" <>
Subject [jira] [Created] (HIVE-6448) Java heap space pasring a query with many union all
Date Mon, 17 Feb 2014 13:20:21 GMT
dima machlin created HIVE-6448:

             Summary: Java heap space pasring a query with many union all
                 Key: HIVE-6448
             Project: Hive
          Issue Type: Bug
          Components: HiveServer2, Query Processor
    Affects Versions: 0.11.0
            Reporter: dima machlin

too many union all statements in a single query require massive amount of memory to process.
Under default HiveServer Xmx (256) we can run ~50 union alls.
It seems to be rather linear as increasing the Xmx to 1024 we can run ~200.
The error is :

java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(
        at java.lang.AbstractStringBuilder.expandCapacity(
        at java.lang.AbstractStringBuilder.append(
        at java.lang.StringBuilder.append(
        at org.apache.hadoop.hive.ql.QueryPlan.getJSONKeyValue(
        at org.apache.hadoop.hive.ql.QueryPlan.getJSONQuery(
        at org.apache.hadoop.hive.ql.QueryPlan.toString(
        at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(

This message was sent by Atlassian JIRA

View raw message