spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: streaming issue
Date Thu, 30 Jul 2015 07:49:01 GMT
What operation are you doing with streaming? Also can you look in the
datanode logs and see whats going on?

Thanks
Best Regards

On Tue, Jul 28, 2015 at 8:18 AM, guoqing0629@yahoo.com.hk <
guoqing0629@yahoo.com.hk> wrote:

> Hi,
> I got a error when running spark streaming as below .
>
> java.lang.reflect.InvocationTargetException
>
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$3.apply(EventLoggingListener.scala:144)
>
> at org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$3.apply(EventLoggingListener.scala:144)
>     at scala.Option.foreach(Option.scala:236)
>
> at org.apache.spark.scheduler.EventLoggingListener.logEvent(EventLoggingListener.scala:144)
>
> at org.apache.spark.scheduler.EventLoggingListener.onUnpersistRDD(EventLoggingListener.scala:175)
>     at org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:50)
>
>
> at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
>
> at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
>
> at org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:53)
>
> at org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:36)
>
> at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:76)
>
> at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61)
>
> at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61)
>
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1618)
>
> at org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:60)
> Caused by: java.io.IOException: All datanodes 10.153.192.159:50010
>  are bad. Aborting...
>
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1137)
>
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:933)
>
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:487)
>
> 15/07/28 02:01:10 ERROR LiveListenerBus: Listener EventLoggingListener threw an exception
> java.lang.reflect.InvocationTargetException
>
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$3.apply(EventLoggingListener.scala:144)
>
> at org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$3.apply(EventLoggingListener.scala:144)
>     at scala.Option.foreach(Option.scala:236)
>
> I had set the ulimit in /etc/security/limits.conf  , but still get the
> same exception . can please some body help me to resolved this issue ?
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 264192
> max locked memory       (kbytes, -l) 32
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 65535
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 10240
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 34816
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
>
> Thanks .
>

Mime
View raw message