hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Teddy Choi (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-15468) Enhance the vectorized execution engine to support complex types
Date Thu, 09 Mar 2017 00:30:38 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902226#comment-15902226
] 

Teddy Choi edited comment on HIVE-15468 at 3/9/17 12:30 AM:
------------------------------------------------------------

[~sershe], thank you. But the following message doesn't match to your explanation. It's not
vectorized because of Select expression for SELECT operator: struct is not supported. Do you
have some idea?

{noformat}
Explain
PLAN VECTORIZATION:
  enabled: true
  enabledConditionsMet: [hive.vectorized.execution.enabled IS true]

STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 depends on stages: Stage-1

STAGE PLANS:
  Stage: Stage-1
    Tez
#### A masked pattern was here ####
      Vertices:
        Map 1 
            Map Operator Tree:
                TableScan
                  alias: orc_create_complex
                  Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE Column stats:
NONE
                  Select Operator
                    expressions: strct (type: struct<a:string,b:string>), mp (type:
map<string,string>), lst (type: array<string>)
                    outputColumnNames: _col0, _col1, _col2
                    Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE Column stats:
NONE
                    File Output Operator
                      compressed: false
                      Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE Column
stats: NONE
                      table:
                          input format: org.apache.hadoop.mapred.SequenceFileInputFormat
                          output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
                          serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
            Execution mode: llap
            LLAP IO: all inputs
            Map Vectorization:
                enabled: true
                enabledConditionsMet: hive.vectorized.use.vectorized.input.format IS true
                inputFileFormats: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
                notVectorizedReason: Select expression for SELECT operator: Data type struct<a:string,b:string>
of Column[strct] not supported
                vectorized: false

  Stage: Stage-0
    Fetch Operator
      limit: -1
      Processor Tree:
        ListSink
{noformat}


was (Author: teddy.choi):
[~sershe], thank you. But the following message doesn't match to your explanation. It's not
vectorized because of Select expression for SELECT operator: struct is not supported. Do you
have some idea?

{{noformat}}
Explain
PLAN VECTORIZATION:
  enabled: true
  enabledConditionsMet: [hive.vectorized.execution.enabled IS true]

STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 depends on stages: Stage-1

STAGE PLANS:
  Stage: Stage-1
    Tez
#### A masked pattern was here ####
      Vertices:
        Map 1 
            Map Operator Tree:
                TableScan
                  alias: orc_create_complex
                  Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE Column stats:
NONE
                  Select Operator
                    expressions: strct (type: struct<a:string,b:string>), mp (type:
map<string,string>), lst (type: array<string>)
                    outputColumnNames: _col0, _col1, _col2
                    Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE Column stats:
NONE
                    File Output Operator
                      compressed: false
                      Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE Column
stats: NONE
                      table:
                          input format: org.apache.hadoop.mapred.SequenceFileInputFormat
                          output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
                          serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
            Execution mode: llap
            LLAP IO: all inputs
            Map Vectorization:
                enabled: true
                enabledConditionsMet: hive.vectorized.use.vectorized.input.format IS true
                inputFileFormats: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
                notVectorizedReason: Select expression for SELECT operator: Data type struct<a:string,b:string>
of Column[strct] not supported
                vectorized: false

  Stage: Stage-0
    Fetch Operator
      limit: -1
      Processor Tree:
        ListSink
{{noformat}}

> Enhance the vectorized execution engine to support complex types
> ----------------------------------------------------------------
>
>                 Key: HIVE-15468
>                 URL: https://issues.apache.org/jira/browse/HIVE-15468
>             Project: Hive
>          Issue Type: Improvement
>          Components: Vectorization
>            Reporter: Chao Sun
>            Assignee: Teddy Choi
>
> Currently Hive's vectorized execution engine only supports scalar types, as documented
here: https://cwiki.apache.org/confluence/display/Hive/Vectorized+Query+Execution.
> To be complete, we should add support for complex types as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message