crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: Weird ClassCastException after updating to CDH 5.4.4 artifacts
Date Fri, 14 Aug 2015 17:42:03 GMT
Agree w/Gabriel that it looks like a Guava version change issue-- did you
do a mvn dependency:tree to see what versions of Guava are coming in to the
jar you're building?

On Fri, Aug 14, 2015 at 7:21 AM, David Ortiz <dortiz@videologygroup.com>
wrote:

> Gabriel,
>
>
>
>      We were on CDH5.3.2 previously.  The client the job was launching
> from is within an Oozie mapper, so it should have the same version of Guava
> available as the job itself since it is sitting on the
> yarn.application.classpath.  What’s weird is the job was running fine on
> the CDH 5.4.4 cluster until I updated the dependencies it is compiled
> against.
>
>
>
> Thanks,
>
>      Dave
>
>
>
> *From:* Gabriel Reid [mailto:gabriel.reid@gmail.com]
> *Sent:* Friday, August 14, 2015 10:12 AM
> *To:* user@crunch.apache.org
> *Subject:* Re: Weird ClassCastException after updating to CDH 5.4.4
> artifacts
>
>
>
> Hi David,
>
>
>
> That stack trace is a little hard to read :-)
>
>
>
> My first guess would be that there is some kind of guava incompatibility
> there, but it might have nothing to do with that. Which version of CDH (and
> guava) were you on before CDH 5.4.4? And can you verify that you've got the
> same version of Guava on the client where you're submitting the job as on
> the machines that are executing the tasks?
>
>
>
> - Gabriel
>
>
>
>
>
> On Thu, Aug 13, 2015 at 12:25 AM David Ortiz <dortiz@videologygroup.com>
> wrote:
>
> Hey,
>
>
>
>       Anyone have any ideas what might cause the following lovely stack
> trace?  Literally the only change is the dependency versions.
> Additionally, I have a few other pipelines reading the same input data sets
> with the same dependencies (down to the parent-pom) and they are running
> without problems.
>
>
>
> Error: java.lang.ClassCastException: cannot assign instance of
> com.google.common.collect.ImmutableList$SerializedForm to field
> org.apache.crunch.types.avro.AvroType.subTypes of type java.util.List in
> instance of org.apache.crunch.types.avro.AvroTableType at
> java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2083)
> at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1996)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at
> java.util.ArrayList.readObject(ArrayList.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at
> java.util.ArrayList.readObject(ArrayList.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at
> java.util.ArrayList.readObject(ArrayList.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at
> java.util.ArrayList.readObject(ArrayList.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at
> java.util.ArrayList.readObject(ArrayList.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at
> java.util.ArrayList.readObject(ArrayList.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at
> org.apache.crunch.util.DistCache.read(DistCache.java:74) at
> org.apache.crunch.impl.mr.run.CrunchTaskContext.<init>(CrunchTaskContext.java:46)
> at org.apache.crunch.impl.mr.run.CrunchMapper.setup(CrunchMapper.java:37)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>
>
>
> Thanks,
>
>      Dave
>
> *This email is intended only for the use of the individual(s) to whom it
> is addressed. If you have received this communication in error, please
> immediately notify the sender and delete the original email.*
>
> *This email is intended only for the use of the individual(s) to whom it
> is addressed. If you have received this communication in error, please
> immediately notify the sender and delete the original email.*
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
View raw message