hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10308) Vectorization execution throws java.lang.IllegalArgumentException: Unsupported complex type: MAP
Date Thu, 21 May 2015 05:59:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553672#comment-14553672
] 

Matt McCline commented on HIVE-10308:
-------------------------------------

[~selinaz] It was part of the huge HIVE-9824 patch (improved vector map join performance).

The change was to VectorExpressionWriterFactory (the class you modified in your patch):

I added this to public static VectorExpressionWriter genVectorExpressionWritable(ObjectInspector
fieldObjInspector):

{noformat}
      case STRUCT:
      case UNION:
      case MAP:
      case LIST:
         return genVectorExpressionWritableEmpty();
{noformat}

and a the new method genVectorExpressionWritableEmpty:

{noformat}
  // For complex types like STRUCT, MAP, etc we do not support, we need a writer that
  // does nothing.  We assume the Vectorizer class has not validated the query to actually
  // try and use the complex types.  They do show up in inputObjInspector[0] and need to be
  // ignored.
  private static VectorExpressionWriter genVectorExpressionWritableEmpty() {
    return new VectorExpressionWriterBase() {
	
      @Override
      public Object writeValue(ColumnVector column, int row)
          throws HiveException {
        return null;
      }
	
      @Override
      public Object setValue(Object row, ColumnVector column, int columnRow)
          throws HiveException {
         return null;
      }
	
      @Override
      public Object initValue(Object ost) throws HiveException {
         return null;
      }
    };
  }	
{noformat}

> Vectorization execution throws java.lang.IllegalArgumentException: Unsupported complex
type: MAP
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-10308
>                 URL: https://issues.apache.org/jira/browse/HIVE-10308
>             Project: Hive
>          Issue Type: Bug
>          Components: Vectorization
>    Affects Versions: 0.14.0, 0.13.1, 1.2.0, 1.1.0
>            Reporter: Selina Zhang
>            Assignee: Matt McCline
>         Attachments: HIVE-10308.1.patch
>
>
> Steps to reproduce:
> {code:sql}
> CREATE TABLE test_orc (a INT, b MAP<INT, STRING>) STORED AS ORC;
> INSERT OVERWRITE TABLE test_orc SELECT 1, MAP(1, "one", 2, "two") FROM src LIMIT 1;
> CREATE TABLE test(key INT) ;
> INSERT OVERWRITE TABLE test SELECT 1 FROM src LIMIT 1;
> set hive.vectorized.execution.enabled=true;
> set hive.auto.convert.join=false;
> select l.key from test l left outer join test_orc r on (l.key= r.a) where r.a is not
null;
> {code}
> Stack trace:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Unsupported complex type: MAP
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory.genVectorExpressionWritable(VectorExpressionWriterFactory.java:456)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory.processVectorInspector(VectorExpressionWriterFactory.java:1191)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorReduceSinkOperator.initializeOp(VectorReduceSinkOperator.java:58)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:362)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:438)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:442)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:198)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message