ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolay Tikhonov <ntikho...@gridgain.com>
Subject Re: Ignite for Spark on YARN Deployment
Date Thu, 09 Jun 2016 14:21:00 GMT
You set wrong value to IGNITE_XML_CONFIG property. The property should
contains path to ignite configuration file. For example

IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml

I think you can comment this line in property file and ignite will start
with default configuration.

On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong <hongmei@kayak.com> wrote:

> Hi nikolai,
>
> Thank you very much for prompt reply!
>
> I did not find the ignite-config.xml file under my ignite home directory(
>  /u/hongmei/apache-ignite/  ).
>
> I find a "default-config.xml" at the path:
> /u/hongmei/apache-ignite/config/default-config.xml
>
> <?xml version="1.0" encoding="UTF-8"?>
>
>
> <!--
>
>   Licensed to the Apache Software Foundation (ASF) under one or more
>
>   contributor license agreements.  See the NOTICE file distributed with
>
>   this work for additional information regarding copyright ownership.
>
>   The ASF licenses this file to You under the Apache License, Version 2.0
>
>   (the "License"); you may not use this file except in compliance with
>
>   the License.  You may obtain a copy of the License at
>
>
>        http://www.apache.org/licenses/LICENSE-2.0
>
>
>   Unless required by applicable law or agreed to in writing, software
>
>   distributed under the License is distributed on an "AS IS" BASIS,
>
>   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>
>   See the License for the specific language governing permissions and
>
>   limitations under the License.
>
> -->
>
>
> <beans xmlns="http://www.springframework.org/schema/beans"
>
>        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>
>        xsi:schemaLocation="
>
>        http://www.springframework.org/schema/beans
>
>        http://www.springframework.org/schema/beans/spring-beans.xsd">
>
>     <!--
>
>         Alter configuration below as needed.
>
>     -->
>
>         <bean id="grid.cfg" class=
> "org.apache.ignite.configuration.IgniteConfiguration">
>
>                 <property name="discoverySpi">
>
>                         <bean class=
> "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>
>                                 <property name="ipFinder">
>
>                                         <bean class=
> "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>                                                 <property name="addresses"
> >
>
>                                                         <list>
>
>                                                                 <!--
>
>
> Explicitly specifying address of a local node to let it start
>
>                                                                 and
> operate normally even if there is no more nodes in the cluster.
>
>                                                                 You can
> also optionally specify an individual port or port range.
>
>                                                                 -->
>
>                                                                 <!--
>
>
> <value>1.2.3.4</value>
>
>                                                                 -->
>
>                                                                 <!--
>
>                                                                         IP
> Address and optional port range of a remote node.
>
>
> You can also optionally specify an individual port and don't set
>
>
> the port range at all.
>
>                                                                 -->
>
>                                                                 <value>
> c5hdpe001.c5.runwaynine.com:47500..47509</value>
>
>                                                                 <value>
> c5hdpe002.c5.runwaynine.com:47500..47509</value>
>
>                                                                 <value>
> c5hdpe003.c5.runwaynine.com:47500..47509</value>
>
>
>                                                         </list>
>
>                                                 </property>
>
>                                         </bean>
>
>                                 </property>
>
>                         </bean>
>
>                 </property>
>
>         </bean>
>
> </beans>
>
>
>
> The "cluster.properties" at the path:
> /u/hongmei/apache-ignite/config/cluster.properties
>
> # The number of nodes in the cluster.
>
> IGNITE_NODE_COUNT=2
>
>
> # The number of CPU Cores for each Apache Ignite node.
>
> IGNITE_RUN_CPU_PER_NODE=1
>
>
> # The number of Megabytes of RAM for each Apache Ignite node.
>
> IGNITE_MEMORY_PER_NODE=2048
>
>
> # The version ignite which will be run on nodes.
>
> IGNITE_VERSION=1.6.0
>
>
> # The hdfs directory which will be used for saving Apache Ignite
> disbributives.
>
> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn
>
>
> #The directory which will be used for saving Apache Ignite
> distributives(copy .jar file to it).
>
> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir
>
>
> #The hdfs path to Apache Ignite config file.
>
> #IGNITE_XML_CONFIG=/user/yarn/ignite/
>
> IGNITE_XML_CONFIG=/user/hongmei/ignite/
>
>
> #The hdfs path to libs which will be added to classpath.
>
> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/
>
>
> #The constraint on slave hosts.
>
> #IGNITE_HOSTENAME_CONSTRAINT=
>
>
> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite
>
>
>
> Thank you very much!!!!
>
> Hongmei
>
>
>
>
>
> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov <ntikhonov@apache.org>
> wrote:
>
>> It seems that your ignite configuration invalid. Could you share ignite-config.xml
>> and /u/hongmei/apache-ignite/config/cluster.properties?
>>
>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong <hongmei@kayak.com> wrote:
>>
>>> Thank you very much Nikolai !
>>>
>>> I found another issue regarding my Ignite YARN Integration:
>>>
>>> I  run Ignite YARN application on one client server machine 'c5hdpe001',
>>> screenshot as the following:
>>> Then I log onto one of the container node and the log is as the
>>> following:
>>>
>>> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar
>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar
>>> /u/hongmei/apache-ignite/config/cluster.properties
>>>
>>> WARNING: Use "yarn jar" to launch YARN applications.
>>>
>>> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service
>>> address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/
>>>
>>> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: Failing
>>> over to rm2
>>>
>>> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application
>>> application_1464374946035_29511
>>>
>>> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main
>>>
>>> INFO: Submitted application. Application id:
>>> application_1464374946035_29511
>>>
>>> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main
>>> *INFO: Application application_1464374946035_29511 is RUNNING.*
>>>
>>>
>>> Then I log onto one of the container node and the log is as the
>>> following: "It failed to start grid".
>>> Any good suggestion? Thank you very much!!!
>>>
>>>
>>> class org.apache.ignite.IgniteException: Failed to instantiate Spring
>>> XML application context
>>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
>>> err=Line 1 in XML document from URL
>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
>>> 1; columnNumber: 1; Content is not allowed in prolog.]
>>>         at
>>> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904)
>>>         at org.apache.ignite.Ignition.start(Ignition.java:350)
>>>         at
>>> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
>>> *Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>>> instantiate Spring XML application context
>>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
>>> err=Line 1 in XML document from URL
>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
>>> 1; columnNumber: 1; Content is not allowed in prolog.]*
>>>         at
>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
>>>         at
>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
>>>         at
>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
>>>         at
>>> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606)
>>>         at
>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807)
>>>         at
>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
>>>         at
>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586)
>>>         at
>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556)
>>>         at org.apache.ignite.Ignition.start(Ignition.java:347)
>>>         ... 1 more
>>> *Caused by:
>>> org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line
>>> 1 in XML document from URL
>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
>>> 1; columnNumber: 1; Content is not allowed in prolog.*
>>>         at
>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398)
>>>         at
>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335)
>>>         at
>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)
>>>         at
>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379)
>>>         ... 9 more
>>> *Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber:
>>> 1; Content is not allowed in prolog.*
>>>         at
>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
>>>         at
>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
>>>         at
>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
>>>         at
>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368)
>>>         at
>>> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1436)
>>>         at
>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:999)
>>>         at
>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
>>>         at
>>> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
>>>         at
>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
>>>         at
>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
>>>         at
>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
>>>         at
>>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>>>         at
>>> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
>>>         at
>>> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
>>>         at
>>> org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)
>>>         at
>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:428)
>>>         at
>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:390)
>>>         ... 12 more
>>> *Failed to start grid: Failed to instantiate Spring XML application
>>> context *[springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/,
>>> err=Line 1 in XML document from URL
>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/]
>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber:
>>> 1; columnNumber: 1; Content is not allowed in prolog.]
>>>
>>>
>>> Hongmei
>>>
>>>
>>> On Thu, Jun 9, 2016 at 6:04 AM, Nikolai Tikhonov <ntikhonov@apache.org>
>>> wrote:
>>>
>>>> *your_address1:47500..47510,your_address2:47500..47510
>>>>> and your_address3:47500..47510 are the YARN master_host address, right?*
>>>>>
>>>>
>>>> No, this addresses hosts on which deploy YARN cluster. For example, you
>>>> have YARN cluster which contains two servers: 10.0.0.1 and 10.0.0.2. In
>>>> this case you will have the following configuration:
>>>>
>>>> ipFinder.setAddresses(Arrays.asList("10.0.0.1:47500..47510",
>>>> "10.0.0.2:47500..47510"));
>>>>
>>>>
>>>
>>
>

Mime
View raw message