hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Zeyliger (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1692) FetchOperator.getInputFormatFromCache hides causal exception
Date Wed, 06 Oct 2010 04:57:30 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918422#action_12918422
] 

Philip Zeyliger commented on HIVE-1692:
---------------------------------------

BTW, to illustrate what a difference 3 characters make, compare debugging the following two
errors:

(no patch)
{noformat}
10/10/05 21:55:39 ERROR CliDriver: Failed with exception java.io.IOException:java.io.IOException:
Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as
specified in mapredWork!
java.io.IOException: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat
as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:271)
	at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:113)
	at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:657)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:131)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat
as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:113)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:214)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:250)
	... 10 more
{noformat}

(patch)
{noformat}

10/10/05 21:54:03 ERROR CliDriver: Failed with exception java.io.IOException:java.io.IOException:
Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as
specified in mapredWork!
java.io.IOException: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat
as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:271)
	at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:113)
	at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:657)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:131)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat
as specified in mapredWork!
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:113)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:214)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:250)
	... 10 more
Caused by: java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
	at org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:109)
	... 12 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
	... 15 more
Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec
not found.
	at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
	at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
	at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:41)
	... 20 more
Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:247)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:907)
	at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:89)
	... 22 more
{noformat}

Obviously, the latter is better, because it gets to the root cause of the problem.  Searching
the user list turned out that I wasn't the first to run into this.

> FetchOperator.getInputFormatFromCache hides causal exception
> ------------------------------------------------------------
>
>                 Key: HIVE-1692
>                 URL: https://issues.apache.org/jira/browse/HIVE-1692
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.7.0
>            Reporter: Philip Zeyliger
>            Priority: Minor
>             Fix For: 0.7.0
>
>         Attachments: HIVE-1692.patch.txt
>
>
> There's a line in FetchOperator.getInputFormatFromCache that catches all exceptions and
re-throws IOException instead, hiding the original cause.  I ran into this, naturally, and
wish to fix it.  Patch below is trivial.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message