hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-718) Load data inpath into a new partition without overwrite does not move the file
Date Fri, 11 Sep 2009 23:07:57 GMT

    [ https://issues.apache.org/jira/browse/HIVE-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754408#action_12754408
] 

Todd Lipcon commented on HIVE-718:
----------------------------------

Namit: here's a trace from a session on hive 0.3.0:

{noformat}
todd@todd-laptop:~$ cat /tmp/insert.txt 
a
b
c
d
todd@todd-laptop:~$ cat /tmp/insert2.txt 
e
f
g
h
todd@todd-laptop:~$ hive
Hive history file=/tmp/todd/hive_job_log_todd_200909111603_978288634.txt
hive> create table tmp_insert_test_p (value string) partitioned by (ds string);
OK
Time taken: 3.865 seconds
hive> load data local inpath '/tmp/insert.txt' into table tmp_insert_test_p partition (ds
= '2009-08-01');
Copying data from file:/tmp/insert.txt
Loading data to table tmp_insert_test_p partition {ds=2009-08-01}
OK
Time taken: 0.672 seconds
hive> select * from tmp_insert_test_p where ds = '2009-08-01';
OK
a       2009-08-01
b       2009-08-01
c       2009-08-01
d       2009-08-01
Time taken: 0.374 seconds
hive> load data local inpath '/tmp/insert2.txt' into table tmp_insert_test_p partition
(ds = '2009-08-01');
Copying data from file:/tmp/insert2.txt
Loading data to table tmp_insert_test_p partition {ds=2009-08-01}
OK
Time taken: 0.261 seconds
hive> select * from tmp_insert_test_p where ds = '2009-08-01';
OK
a       2009-08-01
b       2009-08-01
c       2009-08-01
d       2009-08-01
e       2009-08-01
f       2009-08-01
g       2009-08-01
h       2009-08-01
Time taken: 0.14 seconds
{noformat}

The same session fails on the 0.4 branch:

{noformat}
hive> create table tmp_insert_test_p (value string) partitioned by (ds string);
OK
Time taken: 0.068 seconds
hive> load data local inpath '/tmp/insert.txt' into table tmp_insert_test_p partition (ds
= '2009-08-01');
Copying data from file:/tmp/insert.txt
Loading data to table tmp_insert_test_p partition {ds=2009-08-01}
OK
Time taken: 0.315 seconds
hive> select * from tmp_insert_test_p where ds = '2009-08-01';
OK
Time taken: 0.523 seconds
{noformat}

> Load data inpath into a new partition without overwrite does not move the file
> ------------------------------------------------------------------------------
>
>                 Key: HIVE-718
>                 URL: https://issues.apache.org/jira/browse/HIVE-718
>             Project: Hadoop Hive
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>            Reporter: Zheng Shao
>         Attachments: HIVE-718.1.patch, HIVE-718.2.patch, hive-718.txt
>
>
> The bug can be reproduced as following. Note that it only happens for partitioned tables.
The select after the first load returns nothing, while the second returns the data correctly.
> insert.txt in the current local directory contains 3 lines: "a", "b" and "c".
> {code}
> > create table tmp_insert_test (value string) stored as textfile;
> > load data local inpath 'insert.txt' into table tmp_insert_test;
> > select * from tmp_insert_test;
> a
> b
> c
> > create table tmp_insert_test_p ( value string) partitioned by (ds string) stored
as textfile;
> > load data local inpath 'insert.txt' into table tmp_insert_test_p partition (ds =
'2009-08-01');
> > select * from tmp_insert_test_p where ds= '2009-08-01';
> > load data local inpath 'insert.txt' into table tmp_insert_test_p partition (ds =
'2009-08-01');
> > select * from tmp_insert_test_p where ds= '2009-08-01';
> a       2009-08-01
> b       2009-08-01
> d       2009-08-01
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message