hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yin Huai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
Date Fri, 14 Sep 2012 21:38:08 GMT

     [ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yin Huai updated HIVE-2206:
---------------------------

    Attachment: HIVE-2206.10-r1384442.patch.txt

The patch is ported to the latest trunk (revision 1384442). I tested this patch with an enabled
CorrelationOptimizer (hive.optimize.correlation=true). During the testing, I fixed several
bugs and all tests should be ok except those I explained below. 

In case TestParse, there are 42 queries failed. Since I made several minor changes in SemanticAnalyzer.
Seems those results should be updated. 

In TestCliDriver, auto_join26.q is failed since it is optimized by the optimizer. Considering
I will make the optimizer disabled by default, I will not do any change regarding this query
and its result. 

In TestCliDriver, create_view.q and udaf_percentile_approx.q are two weird queries. If hive.map.aggr=false,
the original trunk will also fail. Seems bug is involved in the trunk. I have sent an email
to dev mailing list regarding create_view.q. For udaf_percentile_approx.q, I have got time
to look at it in detail.

In TestCliDriver, join31.q is failed. For this case, the query should be updated to have "set
hive.optimize.correlation=true". But, since the optimizer is disabled by default, I will not
update this query. 

Also, I got some queries which trunk cannot pass. These are cascade_dbdrop_hadoop20.q, hbase_binary_external_table_queries.q,
hbase_binary_map_queries.q, hbase_binary_storage_queries.q, hbase_joins.q, hbase_ppd_key_range.q,
hbase_pushdown.q, hbase_queries.q, local_mapred_error_cache.q, and TestCase TestHBaseMinimrCliDriver.


I will run all tests again and will fix any bug related to the patch. 

                
> add a new optimizer for query correlation discovery and optimization
> --------------------------------------------------------------------
>
>                 Key: HIVE-2206
>                 URL: https://issues.apache.org/jira/browse/HIVE-2206
>             Project: Hive
>          Issue Type: New Feature
>            Reporter: He Yongqiang
>            Assignee: Yin Huai
>         Attachments: HIVE-2206.10-r1384442.patch.txt, HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt,
HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, HIVE-2206.5.patch.txt,
HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, HIVE-2206.8.r1224646.patch.txt, HIVE-2206.8-r1237253.patch.txt,
testQueries.2.q, YSmartPatchForHive.patch
>
>
> reference:
> http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message