hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-718) Load data inpath into a new partition without overwrite does not move the file
Date Mon, 14 Sep 2009 20:17:10 GMT

    [ https://issues.apache.org/jira/browse/HIVE-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755176#action_12755176
] 

Namit Jain commented on HIVE-718:
---------------------------------

In 0.3, if we insert the same file twice, we get a error 

2009-09-14 13:15:16,422 ERROR exec.MoveTask (SessionState.java:printError(279)) - Failed with
exception checkPaths: /data/users/njain/hive6/hive6/build/ql/test/data/warehouse/tmp_insert_test_p/ds=2009-08-01/kv1.txt
already exists
org.apache.hadoop.hive.ql.metadata.HiveException: checkPaths: /data/users/njain/hive6/hive6/build/ql/test/data/warehouse/tmp_insert_test_p/ds=2009-08-01/kv1.txt
already exists
	at org.apache.hadoop.hive.ql.metadata.Hive.checkPaths(Hive.java:703)
	at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:726)
	at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:522)
	at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:145)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:245)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:176)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:211)
	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:411)
	at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input40(TestCliDriver.java:57)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at junit.framework.TestCase.runTest(TestCase.java:154)
	at junit.framework.TestCase.runBare(TestCase.java:127)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:118)
	at junit.framework.TestSuite.runTest(TestSuite.java:208)
	at junit.framework.TestSuite.run(TestSuite.java:203)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567)


I will stick with the above error for now for 0.4 also.


> Load data inpath into a new partition without overwrite does not move the file
> ------------------------------------------------------------------------------
>
>                 Key: HIVE-718
>                 URL: https://issues.apache.org/jira/browse/HIVE-718
>             Project: Hadoop Hive
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>            Reporter: Zheng Shao
>         Attachments: HIVE-718.1.patch, HIVE-718.2.patch, hive-718.txt
>
>
> The bug can be reproduced as following. Note that it only happens for partitioned tables.
The select after the first load returns nothing, while the second returns the data correctly.
> insert.txt in the current local directory contains 3 lines: "a", "b" and "c".
> {code}
> > create table tmp_insert_test (value string) stored as textfile;
> > load data local inpath 'insert.txt' into table tmp_insert_test;
> > select * from tmp_insert_test;
> a
> b
> c
> > create table tmp_insert_test_p ( value string) partitioned by (ds string) stored
as textfile;
> > load data local inpath 'insert.txt' into table tmp_insert_test_p partition (ds =
'2009-08-01');
> > select * from tmp_insert_test_p where ds= '2009-08-01';
> > load data local inpath 'insert.txt' into table tmp_insert_test_p partition (ds =
'2009-08-01');
> > select * from tmp_insert_test_p where ds= '2009-08-01';
> a       2009-08-01
> b       2009-08-01
> d       2009-08-01
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message