mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pat Ferrel <...@occamsmachete.com>
Subject Re: intermitent unit test error
Date Fri, 20 Feb 2015 18:10:25 GMT
I wonder of it has to do with the number of contexts problem since it always completes with
a partial build.

I’d like to get that in with 1.2.1 this weekend so maybe it will go away.

On Feb 20, 2015, at 10:06 AM, Dmitriy Lyubimov <dlieu.7@gmail.com> wrote:

We had various operational configuration problems with snappy as well so
had to disable it for now completely until somebody has time to figure it
out (which has been like forever)

On Thu, Feb 19, 2015 at 4:26 PM, Pat Ferrel <pat@occamsmachete.com> wrote:

> It seems like after a clean install I fail some test in spark like the one
> below. Then when I build again starting with -rf spark everything passes.
> I’m not sure its always after a clean but happens often.
> 
> Anyone running into this?
> 
> - C = A cbind B, cogroup *** FAILED ***
>  org.apache.spark.SparkException: Job aborted due to stage failure: Task
> 0 in stage 205.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 205.0 (TID 442, localhost): java.io.IOException: PARSING_ERROR(2)
>        org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:78)
>        org.xerial.snappy.SnappyNative.uncompressedLength(Native Method)
>        org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:545)
> 
> org.xerial.snappy.SnappyInputStream.readFully(SnappyInputStream.java:125)
> 
> org.xerial.snappy.SnappyInputStream.readHeader(SnappyInputStream.java:88)
> 
> org.xerial.snappy.SnappyInputStream.<init>(SnappyInputStream.java:58)
> 
> org.apache.spark.io.SnappyCompressionCodec.compressedInputStream(CompressionCodec.scala:128)
> 
> org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:232)
> 
> org.apache.spark.broadcast.TorrentBroadcast.readObject(TorrentBroadcast.scala:169)
>        sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        java.lang.reflect.Method.invoke(Method.java:606)
> 
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
> 
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
> 
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
>        java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> 
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
> 
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
> 
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
>        java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
>        java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> 
> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:62)
> 
> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:87)
> 
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:159)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>        java.lang.Thread.run(Thread.java:745)
> Driver stacktrace:
>  at org.apache.spark.scheduler.DAGScheduler.org
> $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185)
>  at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174)
>  at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173)
>  at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
>  at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173)
>  at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
>  at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
>  at scala.Option.foreach(Option.scala:236)
>  at
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688)
>  ...
> - C = A cbind B, zip
> 
> 


Mime
View raw message