hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2260) ExecDriver::addInputPaths should pass the table properties to the record writer
Date Wed, 13 Jul 2011 22:30:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064916#comment-13064916
] 

Jakob Homan commented on HIVE-2260:
-----------------------------------

resonding to RB comment.
bq. Is it possible to add a testcase? If not, can you update the jira how it is tested?
Adding a testcase isn't possible without something like Mockito, since this is buried deep
within the guts of a private method that calls many methods and creates new instances of classes
from calls to getClass().  Even then it would take about a dozen mocks and be amazingly fragile.
 This code is certainly ripe for refactoring to ease testing.

This was manually tested (and deployed to our cluster).  The manual test is to prune on a
non-existent partition while using an input format like Haivvreo's, which makes use of the
Properties instance.  Without the patch, the call fails.  With the patch, it succeeds with
the behavior as described in {{ExecDriver.java}}:
{code:title=ExecDriver}// If the query references non-existent partitions
// We need to add a empty file, it is not acceptable to change the
// operator tree
// Consider the query:
// select * from (select count(1) from T union all select count(1) from
// T2) x;
// If T is empty and T2 contains 100 rows, the user expects: 0, 100 (2
// rows){code} 

> ExecDriver::addInputPaths should pass the table properties to the record writer
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-2260
>                 URL: https://issues.apache.org/jira/browse/HIVE-2260
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.7.0
>            Reporter: Jakob Homan
>            Assignee: Jakob Homan
>         Attachments: HIVE-2260.patch
>
>
> Currently when ExecDriver encounters a non-existent partition, it creates an empty file
so that the query will be valid (and return 0 results).  However, when it does this and calls
{{getHiveRecordWriter()}}, it creates a new instance of Properties, rather than providing
the Properties associated with the table.
> This causes RecordWriters that pull information from the table through the Properties
to fail (such as [Haivvreo|http://bit.ly/iwEQzJ]).  The RecordWriter should be provided the
table's Properties, as it is in all other cases where it's called.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message