storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [storm] Ethanlm opened a new pull request #3303: [STORM-3667] make exceptions more understandable to submitters when topology submission fails due to incorrect worker max heap size config
Date Mon, 06 Jul 2020 22:38:04 GMT

Ethanlm opened a new pull request #3303:
URL: https://github.com/apache/storm/pull/3303


   ## What is the purpose of the change
   
   https://issues.apache.org/jira/browse/STORM-3667
   
   When submitting a topology failed because of incorrect max worker heap size config, the
exception returned to submitter is not easy to understand: "Internal error processing submitTopology"
   
   
   ## How was the change tested
   
   With 
   ```
   bin/storm jar /home/y/lib64/jars/storm-starter.jar  org.apache.storm.starter.WordCountTopology
-c topology.worker.max.heap.size.mb=100  -c topology.component.resources.onheap.memory.mb=120
wc
   ```
   the exception message is much more clear to submitters now:
   ```
   22:34:43.683 [main] ERROR o.a.s.StormSubmitter - Topology submission exception: Topology
will not be able to be successfully scheduled: Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=100.0
< 120.0 (Largest memory requirement of a component in the topology). Perhaps set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB
to a larger amount
   java.lang.RuntimeException: InvalidTopologyException(msg:Topology will not be able to be
successfully scheduled: Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=100.0 < 120.0 (Largest
memory requirement of a component in the topology). Perhaps set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB
to a larger amount)
   	at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:291)
   	at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:210)
   	at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:173)
   	at org.apache.storm.topology.ConfigurableTopology.submit(ConfigurableTopology.java:119)
   	at org.apache.storm.starter.WordCountTopology.run(WordCountTopology.java:58)
   	at org.apache.storm.topology.ConfigurableTopology.start(ConfigurableTopology.java:68)
   	at org.apache.storm.starter.WordCountTopology.main(WordCountTopology.java:36)
   Caused by: InvalidTopologyException(msg:Topology will not be able to be successfully scheduled:
Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=100.0 < 120.0 (Largest memory requirement of a
component in the topology). Perhaps set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a larger amount)
   	at org.apache.storm.generated.Nimbus$submitTopology_result$submitTopology_resultStandardScheme.read(Nimbus.java:9427)
   	at org.apache.storm.generated.Nimbus$submitTopology_result$submitTopology_resultStandardScheme.read(Nimbus.java:9404)
   	at org.apache.storm.generated.Nimbus$submitTopology_result.read(Nimbus.java:9338)
   	at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
   	at org.apache.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:319)
   	at org.apache.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:303)
   	at org.apache.storm.StormSubmitter.submitTopologyInDistributeMode(StormSubmitter.java:344)
   	at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:279)
   	... 6 more
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message