hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nemon Lou (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-14353) Performance degradation after Projection Pruning in CBO
Date Fri, 29 Jul 2016 01:08:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396893#comment-15396893
] 

Nemon Lou edited comment on HIVE-14353 at 7/29/16 1:08 AM:
-----------------------------------------------------------

||queries||CBO_total_time||CBO_time_in_SelectOP||
|q27|	266.494|80.5 |     
|q7	|   328.259|98.8 |
|q68|	369.159|105 |
|q46|	392.777|91.75|

I just run a few of them because of time limit. The time spent in selectOP is calculated by
adding up total times spent for selectOP  in one executor ,and then divide number of cores.(4
in my case).
Also,I have run q46 without projection pruning.And total time is 266.226,time spent in selectOP
is 0.125 seconds.


was (Author: nemon):
||queries||CBO_total_time||CBO_time_in_SelectOP||
|q27|	266.494|251 |     
|q7	|   328.259|98.8 |
|q68|	369.159|105 |
|q46|	392.777|91.75|

I just run a few of them because of time limit. The time spent in selectOP is calculated by
adding up total times spent for selectOP  in one executor ,and then divide number of cores.(4
in my case).
Also,I have run q46 without projection pruning.And total time is 266.226,time spent in selectOP
is 0.125 seconds.

> Performance degradation  after Projection Pruning in CBO
> --------------------------------------------------------
>
>                 Key: HIVE-14353
>                 URL: https://issues.apache.org/jira/browse/HIVE-14353
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO, Logical Optimizer
>    Affects Versions: 1.2.1
>            Reporter: Nemon Lou
>         Attachments: q46_cbo_no_projection_prune_explain.txt, q46_cbo_projection_prune_explain.rar
>
>
> TPC-DS with factor 1024.
> Hive on Spark. 
> With and without projection prunning,time spent are quite different.
> The way to disable projection prunning : disable HiveRelFieldTrimmer in code and compile
a new jar.
> ||queries||CBO_no_projection_prune||CBO||
> |q27|	160|251 |     
> |q7	|   200|312 |
> |q88|	701|1092|
> |q68|	234|345 |
> |q39|    53|78  |
> |q73|	160|228 |
> |q31|	463|659 |
> |q79|	242|343 |
> |q46|	256|363 |
> |q60|	271|382 |
> |q66|	198|278 |
> |q34|	155|217 |
> |q19|	184|256 |
> |q26|	154|214 |
> |q56|	262|364 |
> |q75|	942|1303|
> |q71|	288|388 |
> |q25|	329|442 |
> |q52|	142|190 |
> |q42|	142|189 |
> |q3	|   139|185 |
> |q98|	153|203 |
> |q89|	187|248 |
> |q58|	264|340 |
> |q43|	127|162 |
> |q32|	174|221 |
> |q96|	156|197 |
> |q70|	320|404 |
> |q29|	499|629 |
> |q18|	266|329 |
> |q21|	76 |92  |
> |q90|	139|165 |



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message