ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Kasnacheev <ilya.kasnach...@gmail.com>
Subject Re: Error running ignite in YARN
Date Wed, 06 Dec 2017 16:35:10 GMT
Hello Juan!

Thank you for the hints! With this, I was able to run Ignite on YARN with
your configuration and my patch.

The patch from IGNITE-7080 was already merged to master by N. Tikhonov, so
it will be included in the next versions.

Note that there are still usability issues. I would be glad if you will
propose improvements for launch process. Right now I think support for
hdfs:// for supplying JAR and .properties files won't hurt. Will allow to
launch Ignite on YARN via "Custom JAR" on EMR. WDYT?


Ilya Kasnacheev

2017-12-06 5:19 GMT+03:00 Juan Rodríguez Hortalá <

> Hi Ilya,
> Thanks a lot for your help, I'll try and the test the patch.
> Regarding how I run this, I wait for the cluster to enter in WAITING state
> and then I ssh to the master node as described in
> https://docs.aws.amazon.com/emr/latest/ManagementGuide/
> emr-connect-master-node-ssh.html, and then I just run the commands there.
> Before running the command I put the files in HDFS. We cannot use a
> bootstrap action here because they run before HDFS starts, but another
> option is using a EMR step for this.
> For the node manager logs, I follow https://docs.aws.
> amazon.com/emr/latest/ManagementGuide/emr-web-interfaces.html to have
> access to the resource manager UI, and then I see the nodes that are
> involved with the YARN application. Then I ssh to some of the slave nodes
> involved with the job from the EMR master, and check the node manager logs
> in /var/log/hadoop-yarn. Also we could configure yarn like suggested in
> https://slider.incubator.apache.org/docs/getting_started.html to have 1
> hour of timeout before cleaning up containers, using
> https://docs.aws.amazon.com/emr/latest/ReleaseGuide/
> emr-configure-apps.html to configure that on EMR. The container files are
> then available in the EMR slaves at /mnt/yarn/
> I'll tell you how it goes when I have tested the patch.
> Thanks a lot!
> Juan
> On Fri, Dec 1, 2017 at 6:43 AM, ilya.kasnacheev <ilya.kasnacheev@gmail.com
> > wrote:
>> Hello once more!
>> I have prepared a patch. I think it won't be hard for you to try it out
>> since you are accustomized to building Ignite. Please try it, tell if it
>> solves the current issue.  ignite-7080.patch
>> <http://apache-ignite-users.70518.x6.nabble.com/file/t1312/
>> ignite-7080.patch>
>> I was trying to set Ignite up on AWS EMR today, without much success. I
>> wonder if you could provide me with some directions.
>> > IGNITE_YARN_JAR=/mnt/ignite/apache-ignite-2.3.0-src/modules/
>> yarn/target/ignite-yarn-2.3.0.jar
>> >  yarn jar ${IGNITE_YARN_JAR} ${IGNITE_YARN_JAR}
>> > /mnt/ignite/ignite_yarn.properties
>> Can you tell me what were you using to launch this command? I have tried
>> executing "Custom JAR" step on EMR cluster, after copying Ignite
>> properties
>> and JAR to /mnt on master manually. So I was informed that an application
>> was started, and is now running, but no indication about execution
>> results.
>> I suspect it hung with the same problem that you outlined (or earlier),
>> but
>> I haven't found a way to view application logs, neither "node manager"
>> logs
>> that you are referring to. Can you please point me where I can watch
>> intermediate execution results?
>> Also there's two data access questions. First, Ignite YARN jar expects two
>> files on local FS, the jar itself, and properties file. Is there a way to
>> make sure that these files are on file system, at predictable locations,
>> when starting EMR cluster? To be able to add "Custom JAR" step to cluster
>> initialization? E.g. specify an s3 bucket to be mounted to master node?
>> Another question is that you are specifying ignite zip and ignite config
>> are
>> on hdfs. How would you put these files to hdfs when cluster is created?
>> Or,
>> for that matter, after it is created? I haven't found any access to HDFS
>> in
>> EMR console, neither any instructions on how to access it.
>> Maybe we should change this process in order to use Ignite YARN with EMR
>> without extensive boilerplate?
>> Regards,
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

View raw message