ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Ignite yarn cluster deployment
Date Wed, 13 Dec 2017 23:58:18 GMT

I think you’re a right person to help here. Please let me know if the documentation has
to be updated.


> On Dec 13, 2017, at 8:53 AM, Andrey Yatsuk <yatsuk@megaputer.ru> wrote:
> Here: https://apacheignite-fs.readme.io/docs/installation-deployment <https://apacheignite-fs.readme.io/docs/installation-deployment>
 I found “Embedded Deployment” section where is it written: 
> Embedded deployment means that Apache Ignite nodes are started inside Apache Spark job
processes and are stopped when job dies. There is no need for additional deployment steps
in this case. Apache Ignite code will be distributed to the worker machines using Apache Spark
deployment mechanism and nodes will be started on all workers as a part of IgniteContext initialization.
> How it can be? Is any examples? Or I need just to start IgniteContext without additional
settings in my app and all will works?
> From: Andrey Yatsuk <mailto:yatsuk@megaputer.ru>
> Sent: 13 декабря 2017 г. 17:15
> To: user@ignite.apache.org <mailto:user@ignite.apache.org>
> Subject: Ignite yarn cluster deployment
> Hi!
> Our application is launched from remotely pc via spark-submit in yarn-cluster mode with
Kerberos keytab and principals by this guide: https://spark.apache.org/docs/latest/running-on-yarn.htmll
<https://spark.apache.org/docs/latest/running-on-yarn.htmll> (I can describe more in
detail if it is required.) The advantages of this approach are that I have my own version
of the spark on any cluster.
> Is it possible to automatically deploy Apache Ignite into a Hadoop YARN cluster without
sftp/ssh only by running a bash-script with HADOOP_CONF_DIR/YARN_CONF_DIR configs,
> Also on this guide: https://apacheignite.readme.io/docs/yarn-deployment <https://apacheignite.readme.io/docs/yarn-deployment>
you need to set up IGNITE_NODE_COUNT. Why you can’t get number of nodes in the cluster from
> Best regards,
> Andrey Yatsuk.

View raw message