hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sohan Jain (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2213) Optimize get_partition_names_ps()
Date Fri, 17 Jun 2011 21:29:47 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051333#comment-13051333
] 

Sohan Jain commented on HIVE-2213:
----------------------------------

I'd also like to point one more thing out.  The previous implementation of get_partitions_ps_with_auth()
did not actually make use of the inputted user name or group name, nor did it set any auth
privileges on the desired partitions.  

This patch adds authentication privileges, which unfortunately slows down get_partitions_ps_with_auth(),
since we have to iterate through all of the partitions and set privileges before returning
them.  What is the desired behavior here?

> Optimize get_partition_names_ps()
> ---------------------------------
>
>                 Key: HIVE-2213
>                 URL: https://issues.apache.org/jira/browse/HIVE-2213
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>            Reporter: Sohan Jain
>            Assignee: Sohan Jain
>         Attachments: HIVE-2213.1.patch, HIVE-2213.3.patch
>
>
> If a table has a large number of partitions, get_partition_names_ps() make take a long
time to execute, because we get all of the partition names from the database.  This is not
very memory efficient, and the operation can be pushed down to the JDO layer without getting
all of the names first.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message