ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Kasnacheev <ilya.kasnach...@gmail.com>
Subject Re: Error running ignite in YARN
Date Wed, 06 Dec 2017 16:35:10 GMT
Hello Juan!

Thank you for the hints! With this, I was able to run Ignite on YARN with
your configuration and my patch.

The patch from IGNITE-7080 was already merged to master by N. Tikhonov, so
it will be included in the next versions.

Note that there are still usability issues. I would be glad if you will
propose improvements for launch process. Right now I think support for
hdfs:// for supplying JAR and .properties files won't hurt. Will allow to
launch Ignite on YARN via "Custom JAR" on EMR. WDYT?

Regards,

-- 
Ilya Kasnacheev

2017-12-06 5:19 GMT+03:00 Juan Rodríguez Hortalá <
juan.rodriguez.hortala@gmail.com>:

> Hi Ilya,
>
> Thanks a lot for your help, I'll try and the test the patch.
>
> Regarding how I run this, I wait for the cluster to enter in WAITING state
> and then I ssh to the master node as described in
> https://docs.aws.amazon.com/emr/latest/ManagementGuide/
> emr-connect-master-node-ssh.html, and then I just run the commands there.
> Before running the command I put the files in HDFS. We cannot use a
> bootstrap action here because they run before HDFS starts, but another
> option is using a EMR step for this.
>
> For the node manager logs, I follow https://docs.aws.
> amazon.com/emr/latest/ManagementGuide/emr-web-interfaces.html to have
> access to the resource manager UI, and then I see the nodes that are
> involved with the YARN application. Then I ssh to some of the slave nodes
> involved with the job from the EMR master, and check the node manager logs
> in /var/log/hadoop-yarn. Also we could configure yarn like suggested in
> https://slider.incubator.apache.org/docs/getting_started.html to have 1
> hour of timeout before cleaning up containers, using
> https://docs.aws.amazon.com/emr/latest/ReleaseGuide/
> emr-configure-apps.html to configure that on EMR. The container files are
> then available in the EMR slaves at /mnt/yarn/
>
>
> I'll tell you how it goes when I have tested the patch.
>
> Thanks a lot!
>
> Juan
>
>
> On Fri, Dec 1, 2017 at 6:43 AM, ilya.kasnacheev <ilya.kasnacheev@gmail.com
> > wrote:
>
>> Hello once more!
>>
>> I have prepared a patch. I think it won't be hard for you to try it out
>> since you are accustomized to building Ignite. Please try it, tell if it
>> solves the current issue.  ignite-7080.patch
>> <http://apache-ignite-users.70518.x6.nabble.com/file/t1312/
>> ignite-7080.patch>
>>
>> I was trying to set Ignite up on AWS EMR today, without much success. I
>> wonder if you could provide me with some directions.
>>
>> > IGNITE_YARN_JAR=/mnt/ignite/apache-ignite-2.3.0-src/modules/
>> yarn/target/ignite-yarn-2.3.0.jar
>> >  yarn jar ${IGNITE_YARN_JAR} ${IGNITE_YARN_JAR}
>> > /mnt/ignite/ignite_yarn.properties
>>
>> Can you tell me what were you using to launch this command? I have tried
>> executing "Custom JAR" step on EMR cluster, after copying Ignite
>> properties
>> and JAR to /mnt on master manually. So I was informed that an application
>> was started, and is now running, but no indication about execution
>> results.
>> I suspect it hung with the same problem that you outlined (or earlier),
>> but
>> I haven't found a way to view application logs, neither "node manager"
>> logs
>> that you are referring to. Can you please point me where I can watch
>> intermediate execution results?
>>
>> Also there's two data access questions. First, Ignite YARN jar expects two
>> files on local FS, the jar itself, and properties file. Is there a way to
>> make sure that these files are on file system, at predictable locations,
>> when starting EMR cluster? To be able to add "Custom JAR" step to cluster
>> initialization? E.g. specify an s3 bucket to be mounted to master node?
>>
>> Another question is that you are specifying ignite zip and ignite config
>> are
>> on hdfs. How would you put these files to hdfs when cluster is created?
>> Or,
>> for that matter, after it is created? I haven't found any access to HDFS
>> in
>> EMR console, neither any instructions on how to access it.
>>
>> Maybe we should change this process in order to use Ignite YARN with EMR
>> without extensive boilerplate?
>>
>> Regards,
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>

Mime
View raw message