hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4684) Query with filter constant on left of "=" and column expression on right does not vectorize
Date Thu, 18 Jul 2013 22:30:49 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712999#comment-13712999
] 

Jitendra Nath Pandey commented on HIVE-4684:
--------------------------------------------

There are two issues here :
1) If the left expression is constant and right expression is a generic function, the query
doesn't vectorize because corresponding vector expressions are missing.
2) If the left expression is constant and right is a column expression, the query vectorizes
to an incorrect expression with column on left, which won't work for non-commutative expressions.

The latest patch includes the missing expressions that addresses (1) and also a one line fix
in VectorizationContext that fixes (2).

                
> Query with filter constant on left of "=" and column expression on right does not vectorize
> -------------------------------------------------------------------------------------------
>
>                 Key: HIVE-4684
>                 URL: https://issues.apache.org/jira/browse/HIVE-4684
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Query Processor
>            Reporter: Eric Hanson
>            Assignee: Jitendra Nath Pandey
>         Attachments: Hive-4684.0.patch, Hive-4684.1.patch, HIVE-4684.1.patch, HIVE-4684.2.patch
>
>
> select dmachineid from factsqlengineam_vec_orc where 1073 = dmachineid + 1;
> Does not go down the vectorization path.
> Output:
> hive> select dmachineid from factsqlengineam_vec_orc where 1073 = dmachineid + 1;
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Validating if vectorized execution is applicable
> Cannot vectorize the plan: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException:
org.apache.hadoop.hiv
> e.ql.exec.vector.expressions.gen.FilterLongScalarEqualLongColumn
> Starting Job = job_201306061504_0038, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201306061504_0038
> Kill Command = c:\Hadoop\hadoop-1.1.0-SNAPSHOT\bin\hadoop.cmd job  -kill job_201306061504_0038
> Hadoop job information for Stage-1: number of mappers: 8; number of reducers: 0
> 2013-06-07 10:25:30,932 Stage-1 map = 0%,  reduce = 0%
> 2013-06-07 10:25:39,953 Stage-1 map = 25%,  reduce = 0%
> 2013-06-07 10:25:42,959 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 8.172 sec
> 2013-06-07 10:25:43,962 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 8.172 sec
> ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message