pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2532) Registered classes fail deserialization in frontend
Date Sat, 25 Feb 2012 01:41:48 GMT

    [ https://issues.apache.org/jira/browse/PIG-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216232#comment-13216232
] 

Ashutosh Chauhan commented on PIG-2532:
---------------------------------------

Awesome. Thanks, Travis for this. Can I request this to be back-ported to 0.8 ?
                
> Registered classes fail deserialization in frontend
> ---------------------------------------------------
>
>                 Key: PIG-2532
>                 URL: https://issues.apache.org/jira/browse/PIG-2532
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Travis Crawford
>            Assignee: Travis Crawford
>         Attachments: PIG-2532-v2.patch, PIG-2532-v3.patch, PIG-2532-v4.patch, PIG-2532.patch,
PIG-253_javax.zip
>
>
> This issue came up while integrating HCatalog with our environment. HCatalog jars are
added to the pig command-line with {{-Dpig.additional.jars}} but fails (exception below).
When added to the pig classpath the error goes away.
> We identified the issue as deserialization using the root class loader, not the context
class loader set when the thread is created. This causes HCatSchema which is serialized into
the context to fail deserialization in the thread.
> {code}
> 2012-02-14 21:55:53,936 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 6017: java.io.IOException:
Deserialization error: org.apache.hcatalog.data.schema.HCatSchema
> 	at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
> 	at org.apache.pig.impl.util.UDFContext.deserialize(UDFContext.java:181)
> 	at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.setupUDFContext(MapRedUtil.java:159)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:229)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> 	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:811)
> 	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
> 	at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
> 	at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigJobControl.mainLoopAction(PigJobControl.java:144)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigJobControl.run(PigJobControl.java:121)
> 	at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.ClassNotFoundException: org.apache.hcatalog.data.schema.HCatSchema
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Class.java:247)
> 	at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:603)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1574)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
> 	at java.util.Hashtable.readObject(Hashtable.java:859)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
> 	at java.util.HashMap.readObject(HashMap.java:1030)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
> 	at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
> 	... 15 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message