hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-23111) MsckPartitionExpressionProxy should filter partitions
Date Tue, 31 Mar 2020 22:35:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-23111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072217#comment-17072217
] 

Prasanth Jayachandran commented on HIVE-23111:
----------------------------------------------

1)
{code:java}
LOG.debug(StringUtils.format("Partition expr: %s", expr)); {code}
can be changed to (removes StringUtils dependency and expr doesn't get converted to string
if debug level is not enabled)
{code:java}
if (LOG.isDebugEnabled()) {
LOG.debug("Partition expr: {}", expr); 
}{code}
same comment for the last set of log lines.

2) If we are doing reverse of makePartExpr() shouldn't we also unescape? Will it be better
to add a utility method in Msck that converts string to partSpec that way it can be unit tested
for different types and expressions?

3)  Can you please add unit test for expression conversion and some tests in TestPartitionManagement
to make sure this expression proxy evaluates correctly? tests against makePartExpr and new
utility that does the reverse (comparing the output) will be useful. 

4) 
{code:java}
if (partition.indexOf(col) == -1){
  isMatch = false;
  break;
} {code}
Can you add a comment/example to describe what match are looking for here? are we looking
for column name match here? if so column "year" and column "month_year" will match when we
look for "year"? 

 

> MsckPartitionExpressionProxy should filter partitions
> -----------------------------------------------------
>
>                 Key: HIVE-23111
>                 URL: https://issues.apache.org/jira/browse/HIVE-23111
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Sam An
>            Assignee: Sam An
>            Priority: Major
>         Attachments: Hive-23111.1.patch
>
>
> Currently MsckPartitionExpressionProxy does not filter partition names, this causes
problem for partition auto discovery. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message