pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Sultan Ericsson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3438) HiveColumnarLoader and partition-key throw java.lang.NoSuchMethodError
Date Thu, 29 Aug 2013 05:37:53 GMT

    [ https://issues.apache.org/jira/browse/PIG-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753328#comment-13753328
] 

Jakob Sultan Ericsson commented on PIG-3438:
--------------------------------------------

I'm running running Cloudera version 4.1.2 which bundles hadoop-2.0.0+552 according to https://ccp.cloudera.com/display/SUPPORT/CDH4+Downloadable+Tarballs--4.1.2.

The only jsp-api.jar is in bigtop and its tomcat. I don't believe that this library in the
classpath when running hadoop or pig.

Could be that some library have bundled the jsp-api.jar-classes inside another jar?

{noformat}
user@devdev:/usr/lib$ find /usr/lib/ -name "*jsp-api*.jar"
/usr/lib/hbase/lib/jsp-api-2.1-6.1.14.jar
/usr/lib/hbase/lib/jsp-api-2.1.jar
/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar
/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar
/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar
/usr/lib/bigtop-tomcat/lib/jsp-api.jar
/usr/lib/hadoop/client-0.20/jsp-api-2.1.jar
/usr/lib/hadoop/lib/jsp-api-2.1.jar
/usr/lib/hadoop/client/jsp-api-2.1.jar
/usr/lib/oozie/libtools/jsp-api-2.1.jar
{noformat}
                
> HiveColumnarLoader and partition-key throw java.lang.NoSuchMethodError
> ----------------------------------------------------------------------
>
>                 Key: PIG-3438
>                 URL: https://issues.apache.org/jira/browse/PIG-3438
>             Project: Pig
>          Issue Type: Bug
>          Components: piggybank
>    Affects Versions: 0.10.0
>         Environment: Linux on Cloudera cdh4.1.2
>            Reporter: Jakob Sultan Ericsson
>
> Using HiveColumnarLoader throw exception when trying to use partition keys in a filter.
> Using this script
> {noformat}
> register /usr/lib/hbase/lib/jsp-api-2.1-6.1.14.jar;
> register /usr/lib/hbase/lib/jsp-2.1-6.1.14.jar;
> register /usr/lib/pig/piggybank.jar;
> register /usr/lib/hive/lib/hive-exec-0.9.0-cdh4.1.2.jar;
> register /usr/lib/hive/lib/hive-common-0.9.0-cdh4.1.2.jar;
> a = LOAD '/user/jakobe/hive/warehouse/jakobtest.db/eventjakobe' USING org.apache.pig.piggybank.storage.HiveColumnarLoader('timestamp
bigint,data array<string>');
> f = FILTER a BY sortid=='17';
> fb = FOREACH f GENERATE *;
> store fb into 'fb17out.dat';
> {noformat}
> As you can see, I have tried to add all libraries that relates to java.lang.NoSuchMethodError.
> {noformat}
> ERROR 2117: Unexpected error when launching map reduce job.
> org.apache.pig.backend.executionengine.ExecException: ERROR 2117: Unexpected error when
launching map reduce job.
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:322)
> 	at org.apache.pig.PigServer.launchPlan(PigServer.java:1275)
> 	at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1260)
> 	at org.apache.pig.PigServer.execute(PigServer.java:1250)
> 	at org.apache.pig.PigServer.executeBatch(PigServer.java:362)
> 	at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
> 	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
> 	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> 	at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> 	at org.apache.pig.Main.run(Main.java:430)
> 	at org.apache.pig.Main.main(Main.java:111)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.RuntimeException: Could not resolve error that occured when launching
map reduce job: java.lang.NoSuchMethodError: javax.el.ExpressionFactory.newInstance()Ljavax/el/ExpressionFactory;
> 	at org.apache.pig.piggybank.storage.partition.PathPartitionHelper.listStatus(PathPartitionHelper.java:183)
> 	at org.apache.pig.piggybank.storage.hiverc.HiveRCInputFormat.listStatus(HiveRCInputFormat.java:91)
> 	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:248)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:273)
> 	at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1014)
> 	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1031)
> 	at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:172)
> 	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:943)
> 	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:896)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
> 	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:896)
> 	at org.apache.hadoop.mapreduce.Job.submit(Job.java:531)
> 	at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
> 	at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
> 	at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
> 	at java.lang.Thread.run(Thread.java:662)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$JobControlThreadExceptionHandler.uncaughtException(MapReduceLauncher.java:631)
> 	at java.lang.Thread.dispatchUncaughtException(Thread.java:1874)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message