beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aviem Zur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-981) Not possible to directly submit a pipeline on spark cluster
Date Wed, 10 May 2017 16:43:04 GMT

    [ https://issues.apache.org/jira/browse/BEAM-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004990#comment-16004990
] 

Aviem Zur commented on BEAM-981:
--------------------------------

Moving this to version 2.1.0 as it is not a blocker for 2.0.0 and can be added later without
breaking existing APIs.

> Not possible to directly submit a pipeline on spark cluster
> -----------------------------------------------------------
>
>                 Key: BEAM-981
>                 URL: https://issues.apache.org/jira/browse/BEAM-981
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-spark
>    Affects Versions: 0.6.0
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Kobi Salant
>             Fix For: 2.1.0
>
>
> It's not possible to directly run a pipeline on the spark runner (for instance using
{{mvn exec:java}}. It fails with:
> {code}
> [appclient-register-master-threadpool-0] INFO org.apache.spark.deploy.client.AppClient$ClientEndpoint
- Connecting to master spark://10.200.118.197:7077...
> [shuffle-client-0] ERROR org.apache.spark.network.client.TransportClient - Failed to
send RPC 6813731522650020739 to /10.200.118.197:7077: java.lang.AbstractMethodError: org.apache.spark.network.protocol.MessageWithHeader.touch(Ljava/lang/Object;)Lio/netty/util/ReferenceCounted;
> java.lang.AbstractMethodError: org.apache.spark.network.protocol.MessageWithHeader.touch(Ljava/lang/Object;)Lio/netty/util/ReferenceCounted;
>         at io.netty.util.ReferenceCountUtil.touch(ReferenceCountUtil.java:73)
>         at io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:107)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:820)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:733)
>         at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:111)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:748)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:740)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:826)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:733)
>         at io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:284)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:748)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:740)
>         at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38)
>         at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1101)
>         at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1148)
>         at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1090)
>         at io.netty.util.concurrent.SingleThreadEventExecutor.safeExecute(SingleThreadEventExecutor.java:451)
>         at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
>         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:401)
>         at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:877)
>         at java.lang.Thread.run(Thread.java:745)
> [appclient-register-master-threadpool-0] WARN org.apache.spark.deploy.client.AppClient$ClientEndpoint
- Failed to connect to master 10.200.118.197:7077
> java.io.IOException: Failed to send RPC 6813731522650020739 to /10.200.118.197:7077:
java.lang.AbstractMethodError: org.apache.spark.network.protocol.MessageWithHeader.touch(Ljava/lang/Object;)Lio/netty/util/ReferenceCounted;
>         at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:239)
>         at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:226)
>         at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
>         at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:507)
>         at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:486)
>         at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
>         at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:129)
>         at io.netty.channel.AbstractChannelHandlerContext.notifyOutboundHandlerException(AbstractChannelHandlerContext.java:845)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:750)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:740)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:826)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:733)
>         at io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:284)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:748)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:740)
>         at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38)
>         at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1101)
>         at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1148)
>         at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1090)
>         at io.netty.util.concurrent.SingleThreadEventExecutor.safeExecute(SingleThreadEventExecutor.java:451)
>         at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
>         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:401)
>         at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:877)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.AbstractMethodError: org.apache.spark.network.protocol.MessageWithHeader.touch(Ljava/lang/Object;)Lio/netty/util/ReferenceCounted;
>         at io.netty.util.ReferenceCountUtil.touch(ReferenceCountUtil.java:73)
>         at io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:107)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:820)
>         at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:733)
>         at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:111)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:748)
>         ... 15 more
> {code}
> It looks like a conflict between the Netty version used in Spark and the one in Beam
(just guessing).
> The workaround is to use {{spark-submit}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message