Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 81DB0740D for ; Tue, 27 Dec 2011 22:00:56 +0000 (UTC) Received: (qmail 42610 invoked by uid 500); 27 Dec 2011 22:00:56 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 42414 invoked by uid 500); 27 Dec 2011 22:00:56 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 42406 invoked by uid 500); 27 Dec 2011 22:00:56 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 42403 invoked by uid 99); 27 Dec 2011 22:00:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Dec 2011 22:00:56 +0000 X-ASF-Spam-Status: No, hits=-2001.3 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Dec 2011 22:00:52 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id 0448912C157 for ; Tue, 27 Dec 2011 22:00:31 +0000 (UTC) Date: Tue, 27 Dec 2011 22:00:31 +0000 (UTC) From: "Bobo Vava (Created) (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: <1800570978.47124.1325023231037.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Created] (HIVE-2683) load data twice from the same hdfs file without overwrite fails with return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org load data twice from the same hdfs file without overwrite fails with return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -------------------------------------------------------------------------------------------------------------------------------- Key: HIVE-2683 URL: https://issues.apache.org/jira/browse/HIVE-2683 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.7.1 Environment: Cloudera VM for Linux Red Hat Reporter: Bobo Vava I try to load data into a hive table twice from the same hdfs file without overwrite option. I would expect second load to append to existing data or at least replace it. I get the following error message instead. Note that, during the operation Hive is renaming the source file to _copy_, which makes us believe it is trying to avoid erasing the existing file while moving and both files (original and renamed) would be made available to Hive. But this is not the case. Example of code end error stack : load data inpath 'hdfs://0.0.0.0/user/qafiles/SRC_FILE_EMP.dat' into table EMP_DELIMITED_FILE_TRUNC_TBL ------------------- Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask river returned: 9. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt Loading data to table default.emp_delimited_file_trunc_tbl Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask Failed with exception null 11/12/13 22:13:55 ERROR exec.MoveTask: Failed with exception null java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:1738) at org.apache.hadoop.hive.ql.metadata.Table.copyFiles(Table.java:542) at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1189) at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:197) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:306) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java :1110) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxService Impl.java:499) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java :886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908 ) at java.lang.Thread.run(Thread.java:619) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask 11/12/13 22:13:55 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask 11/12/13 22:13:55 ERROR beeswax.BeeswaxServiceImpl: Exception while processing query BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt Loading data to table default.emp_delimited_file_trunc_tbl Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask , log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f, handle:QueryHandle(id:8e8ea906-9b5a-4bfe-9946-515249cc523f, log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f)) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:313) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java :1110) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:499) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java :886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908 ) at java.lang.Thread.run(Thread.java:619) 11/12/13 22:13:58 ERROR beeswax.BeeswaxServiceImpl: Caught BeeswaxException BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt Loading data to table default.emp_delimited_file_trunc_tbl Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask , log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f, handle:QueryHandle(id:8e8ea906-9b5a-4bfe-9946-515249cc523f, log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f)) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:313) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1110) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:499) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908 ) at java.lang.Thread.run(Thread.java:619) 11/12/13 22:30:51 ERROR beeswax.BeeswaxServiceImpl: Caught BeeswaxException BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt Loading data to table default.emp_delimited_file_trunc_tbl Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask , log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f, handle:QueryHandle(id:8e8ea906-9b5a-4bfe-9946-515249cc523f, log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f)) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:313) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1110) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:499) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira