hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5348) If both table and partition have same serdes, then we don't even need to check for OI conversions in ObjectInspectorConverters
Date Tue, 01 Oct 2013 05:29:25 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13782643#comment-13782643
] 

Xuefu Zhang commented on HIVE-5348:
-----------------------------------

This seems to be a useful optimization, which should help tables with large number of partitions
of largely the same serde. It seems that the patch also includes the "optimization" that HIVE-5298
tried to address. With comments there in HIVE-5298, I'm still not sure whether the optimization
is valid. The question I had is: whether it's possible that a mapoperator can deals multiple
alias with the same partition set. If so, then the optimization become valid because it only
go thru each partition folder once for all alias. If there can only be one alias, then the
number of loops remains the same. (My simple select query only generates one alias.) An example
will be helpful to understand.

Overall, I feel HIVE-3833 introduced certain uncertainty and performance implications. To
optimize, deep understanding of the topic is essential.

> If both table and partition have same serdes, then we don't even need to check for OI
conversions in ObjectInspectorConverters
> ------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-5348
>                 URL: https://issues.apache.org/jira/browse/HIVE-5348
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Hari Sankar Sivarama Subramaniyan
>            Assignee: Hari Sankar Sivarama Subramaniyan
>         Attachments: HIVE-5348.1.patch.txt
>
>
> Follow-up JIRA for HIVE-5202



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message