hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bobo Vava (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-2683) load data twice from the same hdfs file without overwrite fails with return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
Date Tue, 27 Dec 2011 22:00:31 GMT
load data twice from the same hdfs file without overwrite fails with return code  1 from org.apache.hadoop.hive.ql.exec.MoveTask
--------------------------------------------------------------------------------------------------------------------------------

                 Key: HIVE-2683
                 URL: https://issues.apache.org/jira/browse/HIVE-2683
             Project: Hive
          Issue Type: Bug
          Components: SQL
    Affects Versions: 0.7.1
         Environment: Cloudera VM for Linux Red Hat
            Reporter: Bobo Vava


I try to load data into a hive table twice from the same hdfs file without overwrite option.
I would expect second load to append to existing data or at least replace it. 
I get the following error message instead.

Note that, during the operation Hive is renaming the source file to <original_name>_copy_<nbr>,
which
makes us believe it is trying to avoid erasing the existing file while moving and both files
(original and renamed) would be made available to Hive. But this is not the case.

Example of code end error stack : 

load data  inpath 'hdfs://0.0.0.0/user/qafiles/SRC_FILE_EMP.dat' 
into table EMP_DELIMITED_FILE_TRUNC_TBL 

------------------- 
Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 
1 from org.apache.hadoop.hive.ql.exec.MoveTask 

river returned: 9.  Errors: Hive history 
file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt 
Loading data to table default.emp_delimited_file_trunc_tbl 
Failed with exception null 
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.MoveTask 

Failed with exception null 
11/12/13 22:13:55 ERROR exec.MoveTask: Failed with exception null 
java.lang.NullPointerException 
at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:1738) 
at org.apache.hadoop.hive.ql.metadata.Table.copyFiles(Table.java:542) 
at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1189) 
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:197) 
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130) 
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) 
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063) 
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:306)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499)

at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:337) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java :1110) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxService Impl.java:499)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java :886) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908 ) 
at java.lang.Thread.run(Thread.java:619) 

FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.MoveTask 
11/12/13 22:13:55 ERROR ql.Driver: FAILED: Execution Error, return code 1 
from org.apache.hadoop.hive.ql.exec.MoveTask 
11/12/13 22:13:55 ERROR beeswax.BeeswaxServiceImpl: Exception while 
processing query 
BeeswaxException(message:Driver returned: 9.  Errors: Hive history 
file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt 
Loading data to table default.emp_delimited_file_trunc_tbl 
Failed with exception null 
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.MoveTask 
, log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f, 
handle:QueryHandle(id:8e8ea906-9b5a-4bfe-9946-515249cc523f, 
log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f)) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:313)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499)

at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:337) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java :1110) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:499)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java :886) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908 ) 
at java.lang.Thread.run(Thread.java:619) 
11/12/13 22:13:58 ERROR beeswax.BeeswaxServiceImpl: Caught BeeswaxException 
BeeswaxException(message:Driver returned: 9.  Errors: Hive history 
file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt 
Loading data to table default.emp_delimited_file_trunc_tbl 
Failed with exception null 
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.MoveTask 
, log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f, 
handle:QueryHandle(id:8e8ea906-9b5a-4bfe-9946-515249cc523f, 
log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f)) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:313)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499)

at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:337) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1110) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:499)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908 ) 
at java.lang.Thread.run(Thread.java:619) 
11/12/13 22:30:51 ERROR beeswax.BeeswaxServiceImpl: Caught BeeswaxException 
BeeswaxException(message:Driver returned: 9.  Errors: Hive history 
file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt 
Loading data to table default.emp_delimited_file_trunc_tbl 
Failed with exception null 
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.MoveTask 
, log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f, 
handle:QueryHandle(id:8e8ea906-9b5a-4bfe-9946-515249cc523f, 
log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f)) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:313)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510)

at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499)

at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:337) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1110) 
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:499)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
at java.lang.Thread.run(Thread.java:619) 
 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message