hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-1420) problem with sequence and rcfiles are mixed for null partitions
Date Mon, 21 Jun 2010 05:18:25 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

He Yongqiang updated HIVE-1420:

    Attachment: hive-1420.1.patch

Two changes:

One is using fileformat from partition desc when needing to create an empty file.
this is to fix the bug for 'select src from fileformat_mix_test;'

The other is set oc = null when do closing in ExecMapper. 
And in ExecMapper's close, when see no data came to the mapper (which means an empty file),
just return and do nothing.
this is to fix the bug for  'select count(1) from fileformat_mix_test;' When encountering
empty file, we are not closing MapOperator tree. Hope this is fine. If we do the close, we
will get NullPointerException from deep Hadoop code because of non-valid output collector

> problem with sequence and rcfiles are mixed for null partitions
> ---------------------------------------------------------------
>                 Key: HIVE-1420
>                 URL: https://issues.apache.org/jira/browse/HIVE-1420
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.6.0
>            Reporter: Namit Jain
>            Assignee: He Yongqiang
>             Fix For: 0.6.0, 0.7.0
>         Attachments: hive-1420.1.patch
> drop table foo;
> create table foo (src int, value string) partitioned by (ds string);
> alter table foo	set fileformat Sequencefile;
> insert overwrite table foo partition (ds='1')
> select key, value from src;
> alter table foo	add partition (ds='2');
> alter table foo set fileformat rcfile;
> select count(1) from foo;
> The above testcase fails

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message