hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harish Butani (JIRA)" <>
Subject [jira] [Commented] (HIVE-7324) CBO: provide a mechanism to test CBO features based on table stats only (w/o table data)
Date Tue, 22 Jul 2014 03:18:38 GMT


Harish Butani commented on HIVE-7324:

The issue is MetadataImporter only works on an empty DB, otherwise you get into id issues:
ids imported from export files intersect with ids in the metastore. To get this to work through
QTestUtil I can look into having it react to a flag such that it either Imports the schema
or invokes createSources.
But this still would not get to what you were trying to do: you will only be run these tests
along with other .q tests in one go.

> CBO: provide a mechanism to test CBO features based on table stats only (w/o table data)
> ----------------------------------------------------------------------------------------
>                 Key: HIVE-7324
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Harish Butani
>            Assignee: Harish Butani
>         Attachments: HIVE-7324.1.patch, HIVE-7324.2.patch
> Since lot of the CBO work is focused on planning, it will be nice to be able to run explain
query to test CBO features. TPCDS has a rich enough schema and query set. So the patch loads
a dump TPCDS(Scale 10000) stats.
> 1. TestCBO shows a way to load stats from a dump and run explain on a tpcds query. The
output is currently dumped to Sys.out. This can be improved by hooking to QTestUtil, but hopefully
this is a good start.
> 2. Uncovered couple of issues in the process of testing this:
> a) PartitionPruner fails on 'true' constants. For e.g. you will get an error for 
> {code:sql}
> partCol < 100 AND true
> {code}
> This gets exposed because the predicates coming out of Optiq can contain 'true' predicates.
> b) OpTraitsRulesProcFactory:checkBucketedTable checks that number of files = numBuckets.
This fails because there are no dataFiles. So I have altered it to catch exceptions and assume
bucketMapJoinConvertible = false if an exception is encountered here.
> Uploading with these changes in this patch for now. Will carve them out as separate patches.
> [~ashutoshc], [~hagleitn] can you please take a look. 

This message was sent by Atlassian JIRA

View raw message