hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Kidave (JIRA)" <>
Subject [jira] Updated: (HIVE-1609) Support partition filtering in metastore
Date Thu, 16 Sep 2010 04:55:33 GMT


Ajay Kidave updated HIVE-1609:

    Attachment: hive_1609_3.patch

Thanks for the review comments. I was on vacation for few days, sorry for late response. 
I have attached a new patch changing the parser to ANTLR. Added few more tests to check for
parser issues.

* With new DataNucleus version, the default select query for a object fetch is a select distinct
on all the columns. This does not work on LONGVARCHAR columns since they are not comparable
(in derby). Setting default-fetch-group="false" menas that the distinct would not be applied
for the viewtext columns. These columns would be fetched lazily when required.

* I would like to let HIVE-1539 make the change as required, this current feature requires
only DataNucleus 2.1.1. Since 2.2 is not released yet, we can wait for the 2.2 release before
moving to that or HIVE-1539 can do the required testing before using the unreleased version.

* When HIVE-1626 changes to the custom Stack implementation, it would be a one line import
change for this patch.

* I have run checkstyle and fixed the issues. Only one new issue reported is "Got an exception
- java.lang.RuntimeException: Unable to get class information for MetaException." This is
because checkstyle is not resolving generated classes, this issue is already present for other
files referencing generated classes.

* get_partitions_ps can be updated to use the new filtering method. This can be done separately
since I am not sure of the existing use cases of  get_partitions_ps and the howl use cases
do not require this. Same applies for adding a new listPartitionNamesByFilter function. I
can open a JIRA for these if required.
* Test cases are present for the function added in this patch. If we add the above two, tests
would be required for that along with the patch.

> Support partition filtering in metastore
> ----------------------------------------
>                 Key: HIVE-1609
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Metastore
>            Reporter: Ajay Kidave
>            Assignee: Ajay Kidave
>             Fix For: 0.7.0
>         Attachments: hive_1609.patch, hive_1609_2.patch, hive_1609_3.patch
> The metastore needs to have support for returning a list of partitions based on user
specified filter conditions. This will be useful for tools which need to do partition pruning.
Howl is one such use case. The way partition pruning is done during hive query execution need
not be changed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message