storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul R <rahul8...@gmail.com>
Subject Re: submit topology using NimbusClient
Date Thu, 18 Feb 2016 21:16:37 GMT
I had a similar use case.  I just created a form to upload the jar as a
file upload and execute it on nimbus hosts.

./R

On Thu, Feb 18, 2016 at 11:16 AM, Kazansky, Michael <
michael.kazansky@jpmchase.com> wrote:

> Hi Storm gurus,
>
> I am working on the project which should allow users to submit their
> topology jars to Storm. We successfully using command line but now I need
> to write application which allows user to submit jar through let say
> browser.
>
> Here is what I am trying to do
>
>
>
>            Config conf = new Config();
>
>             TopologyBuilder topoBuilder = new TopologyBuilder();
>
>             conf.put(Config.NIMBUS_HOST, topologyModel.getHost());
>
>             conf.setDebug(true);
>
>             Map stormConf = Utils.readStormConfig();
>
>             stormConf.put(Config.NIMBUS_HOST, topologyModel.getHost());
>
>
>
>             // topology submit
>
>             NimbusClient nimbus = new NimbusClient(stormConf,
> topologyModel.getHost(), null != topologyModel.getNimbusPort() ?
> topologyModel.getNimbusPort() : 6627);
>
>             String uploadedJarLocation =
> StormSubmitter.submitJar(stormConf, topologyModel.getArtifactUrl());
>
>             System.setProperty("storm.jar", uploadedJarLocation.concat("
> ").concat(topologyModel.getMainClass()));
>
>             String jsonConf = JSONValue.toJSONString(stormConf);
>
>             //StormTopology topology = buildToplogy(topologyModel);
>
>             //SubmitOptions options = new SubmitOptions();
>
>             //client.jartransformer.class
>
>
> nimbus.getClient().submitTopology(topologyModel.getTopologyName(),
> uploadedJarLocation, jsonConf, topoBuilder.createTopology());
>
>             success = true;
>
>
>
>
>
> I looked at the storm python script and saw that main class name is
> concatenated to JVM_OPTS
>
> That’s why I added this line
>
>
>
> System.setProperty("storm.jar", uploadedJarLocation.concat("
> ").concat(topologyModel.getMainClass()));
>
>
>
> But when I execute the code in test I only see empty topology without
> spouts and bolts in Storm UI with only one worker and one executor.  Main
> class in the jar has the code to submit spouts and bolts with parallelism 3
> so most definitely main class wasn’t picked up.
>
> Is there any way to implement this command python ${storm_home}/bin/storm
> jar ${dsl_apps_dir}/dist/${jarname} $mainclass in Java besides calling it
> (which is already done) directly?
>
>
>
>
>
> Thanks,
>
> Michael Kazansky
>
>
>
> This communication is for informational purposes only. It is not intended
> as an offer or solicitation for the purchase or sale of any financial
> instrument or as an official confirmation of any transaction. All market
> prices, data and other information are not warranted as to completeness or
> accuracy and are subject to change without notice. Any comments or
> statements made herein do not necessarily reflect those of JPMorgan Chase &
> Co., its subsidiaries and affiliates (collectively, "JPMC"). This
> transmission may contain information that is proprietary, privileged,
> confidential and/or exempt from disclosure under applicable law. If you are
> not the intended recipient, you are hereby notified that any disclosure,
> copying, distribution, or use of the information contained herein
> (including any reliance thereon) is STRICTLY PROHIBITED. If you received
> this transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard copy
> format. Although this transmission and any attachments are believed to be
> free of any virus or other defect that might affect any computer system
> into which it is received and opened, it is the responsibility of the
> recipient to ensure that it is virus free and no responsibility is accepted
> by JPMC for any loss or damage arising in any way from its use. Please note
> that any electronic communication that is conducted within or through
> JPMC's systems is subject to interception, monitoring, review, retention
> and external production in accordance with JPMC's policy and local laws,
> rules and regulations; may be stored or otherwise processed in countries
> other than the country in which you are located; and will be treated in
> accordance with JPMC policies and applicable laws and regulations. Please
> refer to http://www.jpmorgan.com/pages/disclosures for disclosures
> relating to European legal entities.
>

Mime
View raw message