impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Knupp <dkn...@cloudera.com>
Subject Re: Error encounted when doing 'bin/load_data.py -w all'
Date Tue, 16 Aug 2016 17:49:26 GMT
Hi Amos,

I see that you're running 'load_data.py -w all' directly by hand.

Have you tried to run ${IMPALA_HOME}/testdata/bin/create-load-data.sh 
instead? It's a wrapper script around load-data.py. You should be able 
to run it without any additional parameters.

You might give it a go as-is to see if it "just works", but assuming it 
won't, it's probably possible to find the lines in there that pertain to 
LZO-formatted data and simply comment them out. This might be easier 
than editing .csv files in testdata.

Just a suggestion,

--David
> Amos Bird <mailto:amosbird@gmail.com>
> August 16, 2016 at 2:26 AM
> Hmm, building hadoop-lzo trapped me with more errors.
> build-native-non-win:
>      [javah] Exception in thread "main" java.lang.NullPointerException
>      [javah] 	at com.sun.tools.javah.JavahTask.run(JavahTask.java:503)
>      [javah] 	at com.sun.tools.javah.JavahTask.run(JavahTask.java:329)
>      [javah] 	at com.sun.tools.javah.Main.main(Main.java:46)
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 4.620 s
> [INFO] Finished at: 2016-08-16T01:12:28-08:00
> [INFO] Final Memory: 21M/1166M
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (build-native-non-win)
on project hadoop-lzo: An Ant BuildException has occured: compilation failed
> [ERROR] around Ant part ...<javah classpath="${build.classes}" destdir="${build.native}/src/com/hadoop/compression/lzo"
force="yes" verbose="yes">... @ 12:122 in /home/amos/hadoop-lzo/target/antrun/build-build-native-non-win.xml
>
>> Alternatively, you could also to try to only load data for the non-LZO
>> formats.
> I think I should go with masking the lzo format. However, I couldn't
> find a way to only load non-lzo formats. I removed all the lzo
> declarations in the testdata csv files, while loading process still
> complained about the missing of hadoop-lzo.
>
> Alex Behm writes:
>
>> Looks like the root cause is that Hadoop's LZO codec could not be found.
>>
>> You probably need to fetch and build this:
>> https://github.com/twitter/hadoop-lzo
>>
>> bin/impala-config.sh assumes that you've checked out that repo into a
>> specific directory $IMPALA_HOME/../hadoop-lzo
>>
>> Alternatively, you could also to try to only load data for the non-LZO
>> formats.
>>
>> Alex
>>
>> On Mon, Aug 15, 2016 at 7:05 PM, Amos Bird<amosbird@gmail.com>  wrote:
>>
>>> Hi Alex,
>>> Thanks for the reply! I have indeed get them all up and running. Here is
>>> the jps list:
>>>
>>> 13184
>>> 12833 LlamaAMMain
>>> 14081 HQuorumPeer
>>> 40870 Jps
>>> 11302 DataNode
>>> 15081 RunJar
>>> 15722 RunJar
>>> 13259
>>> 11980 NodeManager
>>> 11244 DataNode
>>> 14700 HRegionServer
>>> 11278 NameNode
>>> 14353 HRegionServer
>>> 11218 DataNode
>>> 11923 NodeManager
>>> 11955 ResourceManager
>>> 12982 Bootstrap
>>> 14166 HMaster
>>> 11896 NodeManager
>>> 16089 RunJar
>>> 14527 HRegionServer
>>>
>>> I also played around a few SQLs inside impala. It works fine. In fact I
>>> was able to create the test data before. I have no clue what happened in
>>> the newest impala build.
>>>
>>>
>>> Alex Behm writes:
>>>
>>>> Hi Amos!
>>>>
>>>> load-data.py assumes that you have a running cluster. You need to first
>>> get
>>>> these working:
>>>> testdata/bin/run-all.sh
>>>> bin/start-impala-cluster.py
>>>>
>>>> The first command starts all dependent services like HDFS, YARN, Hive
>>>> Metastore, Hive HS2, etc.
>>>> The second command starts an Impala mini-cluster with 3 nodes. This
>>> command
>>>> assumes all dependent services are already running/
>>>>
>>>> Hope it helps!
>>>>
>>>> Alex
>>>>
>>>> On Mon, Aug 15, 2016 at 5:20 AM, Amos Bird<amosbird@gmail.com>  wrote:
>>>>
>>>>> I was trying to build a new test warehouse. After successfully running
>>>>> 'bin/create_testdata.sh', I did 'bin/load_data.py -w all'.
>>> Unfortunately it
>>>>> ended up with this:
>>>>>
>>>>> ERROR : Job Submission failed with exception 'java.io.IOException(java.
>>>>> util.concurrent.ExecutionException: java.io.IOException: Cannot create
>>> an
>>>>> instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat
>>> as
>>>>> specified in mapredWork!)'
>>>>> java.io.IOException: java.util.concurrent.ExecutionException:
>>>>> java.io.IOException: Cannot create an instance of InputFormat class
>>>>> org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
>>>>>    at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(
>>>>> CombineHiveInputFormat.java:544)
>>>>>    at org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(
>>>>> JobSubmitter.java:332)
>>>>>    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(
>>>>> JobSubmitter.java:324)
>>>>>    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(
>>>>> JobSubmitter.java:200)
>>>>>    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
>>>>>    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
>>>>>    at java.security.AccessController.doPrivileged(Native Method)
>>>>>    at javax.security.auth.Subject.doAs(Subject.java:422)
>>>>>    at org.apache.hadoop.security.UserGroupInformation.doAs(
>>>>> UserGroupInformation.java:1693)
>>>>>    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
>>>>>    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:578)
>>>>>    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:573)
>>>>>    at java.security.AccessController.doPrivileged(Native Method)
>>>>>    at javax.security.auth.Subject.doAs(Subject.java:422)
>>>>>    at org.apache.hadoop.security.UserGroupInformation.doAs(
>>>>> UserGroupInformation.java:1693)
>>>>>    at org.apache.hadoop.mapred.JobClient.submitJobInternal(
>>>>> JobClient.java:573)
>>>>>    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:564)
>>>>>    at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(
>>>>> ExecDriver.java:430)
>>>>>    at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(
>>>>> MapRedTask.java:137)
>>>>>    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
>>>>>    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(
>>>>> TaskRunner.java:100)
>>>>>    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1782)
>>>>>    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1539)
>>>>>    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1318)
>>>>>    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1127)
>>>>>    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1120)
>>>>>    at org.apache.hive.service.cli.operation.SQLOperation.
>>>>> runQuery(SQLOperation.java:191)
>>>>>    at org.apache.hive.service.cli.operation.SQLOperation.access$
>>>>> 100(SQLOperation.java:79)
>>>>>    at org.apache.hive.service.cli.operation.SQLOperation$2$1.
>>>>> run(SQLOperation.java:245)
>>>>>    at java.security.AccessController.doPrivileged(Native Method)
>>>>>    at javax.security.auth.Subject.doAs(Subject.java:422)
>>>>>    at org.apache.hadoop.security.UserGroupInformation.doAs(
>>>>> UserGroupInformation.java:1693)
>>>>>    at org.apache.hive.service.cli.operation.SQLOperation$2.run(
>>>>> SQLOperation.java:258)
>>>>>    at java.util.concurrent.Executors$RunnableAdapter.
>>>>> call(Executors.java:511)
>>>>>    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>    at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>>> ThreadPoolExecutor.java:1142)
>>>>>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>>> ThreadPoolExecutor.java:617)
>>>>>    at java.lang.Thread.run(Thread.java:745)
>>>>> Caused by: java.util.concurrent.ExecutionException:
>>> java.io.IOException:
>>>>> Cannot create an instance of InputFormat class org.apache.hadoop.mapred.
>>> TextInputFormat
>>>>> as specified in mapredWork!
>>>>>    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>>>>>    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>>>>>    at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(
>>>>> CombineHiveInputFormat.java:532)
>>>>>    ... 37 more
>>>>> Caused by: java.io.IOException: Cannot create an instance of InputFormat
>>>>> class org.apache.hadoop.mapred.TextInputFormat as specified in
>>> mapredWork!
>>>>>    at org.apache.hadoop.hive.ql.io.HiveInputFormat.
>>> getInputFormatFromCache(
>>>>> HiveInputFormat.java:211)
>>>>>    at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$
>>>>> CheckNonCombinablePathCallable.call(CombineHiveInputFormat.java:111)
>>>>>    at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$
>>>>> CheckNonCombinablePathCallable.call(CombineHiveInputFormat.java:88)
>>>>>    ... 4 more
>>>>> Caused by: java.lang.RuntimeException: Error in configuring object
>>>>>    at org.apache.hadoop.util.ReflectionUtils.setJobConf(
>>>>> ReflectionUtils.java:109)
>>>>>    at org.apache.hadoop.util.ReflectionUtils.setConf(
>>>>> ReflectionUtils.java:75)
>>>>>    at org.apache.hadoop.util.ReflectionUtils.newInstance(
>>>>> ReflectionUtils.java:133)
>>>>>    at org.apache.hadoop.hive.ql.io.HiveInputFormat.
>>> getInputFormatFromCache(
>>>>> HiveInputFormat.java:203)
>>>>>    ... 6 more
>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>    at sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>> NativeMethodAccessorImpl.java:62)
>>>>>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>    at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>    at org.apache.hadoop.util.ReflectionUtils.setJobConf(
>>>>> ReflectionUtils.java:106)
>>>>>    ... 9 more
>>>>> Caused by: java.lang.IllegalArgumentException: Compression codec
>>>>> com.hadoop.compression.lzo.LzoCodec not found.
>>>>>    at org.apache.hadoop.io.compress.CompressionCodecFactory.
>>>>> getCodecClasses(CompressionCodecFactory.java:135)
>>>>>    at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>
>>>>> (CompressionCodecFactory.java:175)
>>>>>    at org.apache.hadoop.mapred.TextInputFormat.configure(
>>>>> TextInputFormat.java:45)
>>>>>    ... 14 more
>>>>> Caused by: java.lang.ClassNotFoundException: Class
>>>>> com.hadoop.compression.lzo.LzoCodec not found
>>>>>    at org.apache.hadoop.conf.Configuration.getClassByName(
>>>>> Configuration.java:2105)
>>>>>    at org.apache.hadoop.io.compress.CompressionCodecFactory.
>>>>> getCodecClasses(CompressionCodecFactory.java:128)
>>>>>    ... 16 more
>>>>>
>>>>> ERROR : FAILED: Execution Error, return code 1 from
>>>>> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>>>>> INFO  : Completed executing command(queryId=amos_
>>>>> 20160815034646_1d786772-c41e-4804-9d3c-dc768656ca3a); Time taken: 0.475
>>>>> seconds
>>>>> Error: Error while processing statement: FAILED: Execution Error, return
>>>>> code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>>>>> (state=08S01,code=1)
>>>>> java.sql.SQLException: Error while processing statement: FAILED:
>>> Execution
>>>>> Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>>>>>    at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
>>>>>    at org.apache.hive.beeline.Commands.executeInternal(
>>> Commands.java:893)
>>>>>    at org.apache.hive.beeline.Commands.execute(Commands.java:1079)
>>>>>    at org.apache.hive.beeline.Commands.sql(Commands.java:976)
>>>>>    at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1089)
>>>>>    at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:921)
>>>>>    at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:899)
>>>>>    at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:841)
>>>>>    at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(
>>>>> BeeLine.java:482)
>>>>>    at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465)
>>>>>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>    at sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>> NativeMethodAccessorImpl.java:62)
>>>>>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>    at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>>>>>    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>>>>>
>>>>> It seems like a mr framework is needed to be running, but
>>>>> 'testdata/bin/run-all.sh' doesn't start it.
>>>>>
>>>>> Any help is much appreciated.
>>>>>
>>>>> regards,
>>>>> Amos.
>>>>>
>>>>>
>>>>>
>
> Alex Behm <mailto:alex.behm@cloudera.com>
> August 15, 2016 at 11:57 PM
> Looks like the root cause is that Hadoop's LZO codec could not be found.
>
> You probably need to fetch and build this:
> https://github.com/twitter/hadoop-lzo
>
> bin/impala-config.sh assumes that you've checked out that repo into a
> specific directory $IMPALA_HOME/../hadoop-lzo
>
> Alternatively, you could also to try to only load data for the non-LZO
> formats.
>
> Alex
>
>
> Amos Bird <mailto:amosbird@gmail.com>
> August 15, 2016 at 7:05 PM
> Hi Alex,
> Thanks for the reply! I have indeed get them all up and running. Here is
> the jps list:
>
> 13184
> 12833 LlamaAMMain
> 14081 HQuorumPeer
> 40870 Jps
> 11302 DataNode
> 15081 RunJar
> 15722 RunJar
> 13259
> 11980 NodeManager
> 11244 DataNode
> 14700 HRegionServer
> 11278 NameNode
> 14353 HRegionServer
> 11218 DataNode
> 11923 NodeManager
> 11955 ResourceManager
> 12982 Bootstrap
> 14166 HMaster
> 11896 NodeManager
> 16089 RunJar
> 14527 HRegionServer
>
> I also played around a few SQLs inside impala. It works fine. In fact I
> was able to create the test data before. I have no clue what happened in
> the newest impala build.
>
>
>
> Alex Behm <mailto:alex.behm@cloudera.com>
> August 15, 2016 at 4:17 PM
> Hi Amos!
>
> load-data.py assumes that you have a running cluster. You need to 
> first get
> these working:
> testdata/bin/run-all.sh
> bin/start-impala-cluster.py
>
> The first command starts all dependent services like HDFS, YARN, Hive
> Metastore, Hive HS2, etc.
> The second command starts an Impala mini-cluster with 3 nodes. This 
> command
> assumes all dependent services are already running/
>
> Hope it helps!
>
> Alex
>
>
> Amos Bird <mailto:amosbird@gmail.com>
> August 15, 2016 at 5:20 AM
> I was trying to build a new test warehouse. After successfully running 
> 'bin/create_testdata.sh', I did 'bin/load_data.py -w all'. 
> Unfortunately it ended up with this:
>
> ERROR : Job Submission failed with exception 
> 'java.io.IOException(java.util.concurrent.ExecutionException: 
> java.io.IOException: Cannot create an instance of InputFormat class 
> org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!)'
> java.io.IOException: java.util.concurrent.ExecutionException: 
> java.io.IOException: Cannot create an instance of InputFormat class 
> org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
> at 
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:544)
> at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:332)
> at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:324)
> at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:578)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:573)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at 
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:573)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:564)
> at 
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:430)
> at 
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1782)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1539)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1318)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1127)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1120)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:191)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:79)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:245)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:258)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException: 
> java.io.IOException: Cannot create an instance of InputFormat class 
> org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at 
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:532)
> ... 37 more
> Caused by: java.io.IOException: Cannot create an instance of 
> InputFormat class org.apache.hadoop.mapred.TextInputFormat as 
> specified in mapredWork!
> at 
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getInputFormatFromCache(HiveInputFormat.java:211)
> at 
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CheckNonCombinablePathCallable.call(CombineHiveInputFormat.java:111)
> at 
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CheckNonCombinablePathCallable.call(CombineHiveInputFormat.java:88)
> ... 4 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> at 
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
> at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> at 
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getInputFormatFromCache(HiveInputFormat.java:203)
> ... 6 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: Compression codec 
> com.hadoop.compression.lzo.LzoCodec not found.
> at 
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
> at 
> org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
> at 
> org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
> ... 14 more
> Caused by: java.lang.ClassNotFoundException: Class 
> com.hadoop.compression.lzo.LzoCodec not found
> at 
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2105)
> at 
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
> ... 16 more
>
> ERROR : FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> INFO : Completed executing 
> command(queryId=amos_20160815034646_1d786772-c41e-4804-9d3c-dc768656ca3a); 
> Time taken: 0.475 seconds
> Error: Error while processing statement: FAILED: Execution Error, 
> return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 
> (state=08S01,code=1)
> java.sql.SQLException: Error while processing statement: FAILED: 
> Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
> at org.apache.hive.beeline.Commands.executeInternal(Commands.java:893)
> at org.apache.hive.beeline.Commands.execute(Commands.java:1079)
> at org.apache.hive.beeline.Commands.sql(Commands.java:976)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1089)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:921)
> at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:899)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:841)
> at 
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>
> It seems like a mr framework is needed to be running, but 
> 'testdata/bin/run-all.sh' doesn't start it.
>
> Any help is much appreciated.
>
> regards,
> Amos.
>
>

-- 
David Knupp
Software Engineer
Cloudera
415-312-1049
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message