hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1328) make mapred.input.dir.recursive work for select *
Date Thu, 29 Apr 2010 14:48:56 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862217#action_12862217
] 

Edward Capriolo commented on HIVE-1328:
---------------------------------------

I find external partitions to be pretty badly broken now. I am circling around one or two
other bugs in them, that I am about to report. Users (including myself) are frustrated beause
rather then working with data they have to work around bugs like HIVE-1318. I understand everyone
has their own priorities. Call it what you will (inconsistancy/feature) we are adding to the
capability of external tables while current features do not even work well. 

In particular HIVE-1318 is brutal. When working with my data I can make no assumptions when
querying. I have to do all types of shell scripting to ensure that partitions exist before
I query them, adding extra where clauses to carefully select ranges of partitions. 

If you are using external partitions at facebook, I wonder how you work around HIVE-1318,
and I am also curious if you experience HIVE-1303 or is this just something in my environment.
The handfull of users I have constantly have issues, does everyone there just 'suck it up'?

> make mapred.input.dir.recursive work for select *
> -------------------------------------------------
>
>                 Key: HIVE-1328
>                 URL: https://issues.apache.org/jira/browse/HIVE-1328
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.6.0
>            Reporter: John Sichi
>            Assignee: John Sichi
>             Fix For: 0.6.0
>
>
> For the script below, we would like the behavior from MAPREDUCE-1501 to apply so that
the select * returns two rows instead of none.
> create table fact_daily(x int)
> partitioned by (ds string);
> create table fact_tz(x int)
> partitioned by (ds string, hr string, gmtoffset string);
> alter table fact_tz 
> add partition (ds='2010-01-03', hr='1', gmtoffset='-8');
> insert overwrite table fact_tz
> partition (ds='2010-01-03', hr='1', gmtoffset='-8')
> select key+11 from src where key=484;
> alter table fact_tz 
> add partition (ds='2010-01-03', hr='2', gmtoffset='-7');
> insert overwrite table fact_tz
> partition (ds='2010-01-03', hr='2', gmtoffset='-7')
> select key+12 from src where key=484;
> alter table fact_daily
> set tblproperties('EXTERNAL'='TRUE');
> alter table fact_daily
> add partition (ds='2010-01-03')
> location '/user/hive/warehouse/fact_tz/ds=2010-01-03';
> set mapred.input.dir.recursive=true;
> select * from fact_daily where ds='2010-01-03';

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message