hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Teja Chilukuri (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-16398) [Hive on Tez] Jar Localization issues
Date Thu, 06 Apr 2017 07:22:41 GMT

     [ https://issues.apache.org/jira/browse/HIVE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ravi Teja Chilukuri updated HIVE-16398:
---------------------------------------
    Description: 
The jar localization fails with the following error in these cases

1) If a jar is already present in the classpath and is added again via aux.jars or using add
jar command by the user.
2) If the jar is missing- Ideally it should ignore or either throw a error/warning stating
jar not found.
3) When a  "*" is present in some of the classpath entries, it will try to replicate the same
* path on hdfs and fails.  



Error:
{noformat}
Total jobs = 1
Launching Job 1 out of 1
Exception in thread "Thread-17" java.lang.RuntimeException: java.io.IOException: Previous
writer likely failed to write hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
Failing because I am unlikely to write too.
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:617)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:526)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78)
Caused by: java.io.IOException: Previous writer likely failed to write hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
Failing because I am unlikely to write too.
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:1020)
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:900)
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:843)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:451)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:247)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:222)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:189)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:612)
	... 2 more
FAILED: Execution Error, return code -1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
{noformat}

  was:
The jar localization fails with the following error in these cases

1) If a jar is already present in the classpath and is added again via aux.jars or using add
jar command by the user.
2) If the jar is missing- Ideally it should ignore or either throw a error/warning stating
jar not found.
3) When a  "*" is present in some of the classpath entries, it will try to replicate the same
* path on hdfs and fails.  



Error:
Total jobs = 1
Launching Job 1 out of 1
Exception in thread "Thread-17" java.lang.RuntimeException: java.io.IOException: Previous
writer likely failed to write hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
Failing because I am unlikely to write too.
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:617)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:526)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78)
Caused by: java.io.IOException: Previous writer likely failed to write hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
Failing because I am unlikely to write too.
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:1020)
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:900)
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:843)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:451)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:247)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:222)
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:189)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:612)
	... 2 more
FAILED: Execution Error, return code -1 from org.apache.hadoop.hive.ql.exec.tez.TezTask


> [Hive on Tez] Jar Localization issues
> -------------------------------------
>
>                 Key: HIVE-16398
>                 URL: https://issues.apache.org/jira/browse/HIVE-16398
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: 2.1.0
>            Reporter: Ravi Teja Chilukuri
>            Priority: Minor
>
> The jar localization fails with the following error in these cases
> 1) If a jar is already present in the classpath and is added again via aux.jars or using
add jar command by the user.
> 2) If the jar is missing- Ideally it should ignore or either throw a error/warning stating
jar not found.
> 3) When a  "*" is present in some of the classpath entries, it will try to replicate
the same * path on hdfs and fails.  
> Error:
> {noformat}
> Total jobs = 1
> Launching Job 1 out of 1
> Exception in thread "Thread-17" java.lang.RuntimeException: java.io.IOException: Previous
writer likely failed to write hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
Failing because I am unlikely to write too.
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:617)
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:526)
> 	at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78)
> Caused by: java.io.IOException: Previous writer likely failed to write hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
Failing because I am unlikely to write too.
> 	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:1020)
> 	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:900)
> 	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:843)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:451)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:247)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:222)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:189)
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:612)
> 	... 2 more
> FAILED: Execution Error, return code -1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message