storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (STORM-395) Using SaslTransportPlugin, fails to submit topology for jar file large than 3K
Date Tue, 08 Jul 2014 22:16:09 GMT

    [ https://issues.apache.org/jira/browse/STORM-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055603#comment-14055603
] 

ASF GitHub Bot commented on STORM-395:
--------------------------------------

GitHub user kishorvpatil opened a pull request:

    https://github.com/apache/incubator-storm/pull/180

    [STORM-395] Use TBinaryProtocol without max buffer

    For storm jar larger than 3K, the SaslTransportPlugin causes [THRIFT-1665]( https://issues.apache.org/jira/browse/THRIFT-1665)
TBinaryProtocol: exceeded message length raises generic TException. This is fixed in thirft
0.9 version. Since we are using 0.7.0 Here is using TBinaryProtocol without max buffer size.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kishorvpatil/incubator-storm thrift-buffersize-patch-security

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-storm/pull/180.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #180
    
----
commit ae5d0c312d170c883e9758767808e1c8d33f58c8
Author: Kishor Patil <kpatil@yahoo-inc.com>
Date:   2014-07-08T22:08:33Z

    Use TBinaryProtocol without max buffer

----


> Using SaslTransportPlugin, fails to submit topology for jar file large than 3K
> ------------------------------------------------------------------------------
>
>                 Key: STORM-395
>                 URL: https://issues.apache.org/jira/browse/STORM-395
>             Project: Apache Storm (Incubating)
>          Issue Type: Bug
>            Reporter: Kishor Patil
>              Labels: security
>
> 669  [main] INFO  backtype.storm.security.auth.AuthUtils - Got AutoCreds []
> 847  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar /home/kpatil/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
to assigned location: storm-local/nimbus/inbox/stormjar-3bf4f7e5-3f9d-487f-9daa-12d8f4202440.jar
> Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException
> 	at backtype.storm.StormSubmitter.submitJar(StormSubmitter.java:371)
> 	at backtype.storm.StormSubmitter.submitJar(StormSubmitter.java:315)
> 	at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:215)
> 	at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> 	at storm.starter.WordCountTopology.main(WordCountTopology.java:78)
> Caused by: org.apache.thrift7.transport.TTransportException
> 	at org.apache.thrift7.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift7.transport.TSaslTransport.readLength(TSaslTransport.java:345)
> 	at org.apache.thrift7.transport.TSaslTransport.readFrame(TSaslTransport.java:422)
> 	at org.apache.thrift7.transport.TSaslTransport.read(TSaslTransport.java:404)
> 	at org.apache.thrift7.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
> 	at org.apache.thrift7.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift7.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift7.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift7.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at backtype.storm.generated.Nimbus$Client.recv_uploadChunk(Nimbus.java:419)
> 	at backtype.storm.generated.Nimbus$Client.uploadChunk(Nimbus.java:405)
> 	at backtype.storm.StormSubmitter.submitJar(StormSubmitter.java:360)
> 	... 4 more
> And on nimbus side:
> 2014-07-08 17:54:08 o.a.t.s.TThreadPoolServer [ERROR] Thrift error occurred during processing
of message.
> org.apache.thrift7.TException: Message length exceeded: 307200
> 	at org.apache.thrift7.protocol.TBinaryProtocol.checkReadLength(TBinaryProtocol.java:393)
~[storm-core-0.9.2-incubating-security.jar:0.9.2-incubating-security]
> 	at org.apache.thrift7.protocol.TBinaryProtocol.readBinary(TBinaryProtocol.java:363)
~[storm-core-0.9.2-incubating-security.jar:0.9.2-incubating-security]
> 	at backtype.storm.generated.Nimbus$uploadChunk_args.read(Nimbus.java:9555) ~[storm-core-0.9.2-incubating-security.jar:0.9.2-incubating-security]
> 	at org.apache.thrift7.ProcessFunction.process(ProcessFunction.java:21) ~[storm-core-0.9.2-incubating-security.jar:0.9.2-incubating-security]
> 	at org.apache.thrift7.TBaseProcessor.process(TBaseProcessor.java:34) ~[storm-core-0.9.2-incubating-security.jar:0.9.2-incubating-security]
> 	at backtype.storm.security.auth.SaslTransportPlugin$TUGIWrapProcessor.process(SaslTransportPlugin.java:134)
~[storm-core-0.9.2-incubating-security.jar:0.9.2-incubating-security]
> 	at org.apache.thrift7.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
~[storm-core-0.9.2-incubating-security.jar:0.9.2-incubating-security]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_60]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_60]
> 	at java.lang.Thread.run(Thread.java:745) [na:1.7.0_60]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message