ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ignite_user2016 <rishiyag...@gmail.com>
Subject Re: Ignite errors in log
Date Tue, 18 Apr 2017 21:36:33 GMT
Thanks, Andrew.

I enabled the ipv4 stack option and see if that resolves our problem.

I will keep you posted, thank you for all your help.


On Tue, Apr 18, 2017 at 1:41 PM, Andrew Mashenkov [via Apache Ignite Users]
<ml-node+s70518n12044h18@n6.nabble.com> wrote:

> Hi Rishi,
>
> Would you please check if both nodes has same either
> java.net.preferIPv4Stack=true or java.net.preferIPv6Stack=true option?
> "remote_host/remote_host:47102" looks weird. Would you also check if all
> dns names are correctly resolved on both nodes?
>
> On Tue, Apr 18, 2017 at 9:29 PM, Rishi Yagnik <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=12044&i=0>> wrote:
>
>> I checked the port and all ports are open between 2 ignite instances.
>>
>> There is something more going on, will provide the log soon.
>>
>> On Tue, Apr 18, 2017 at 12:00 PM, ignite_user2016 <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=12044&i=1>> wrote:
>>
>>> we use Ignite 1.7, yes application is communicating between hosts and
>>> client application.
>>>
>>> I will provide the logs soon.
>>>
>>> Thanks,
>>>
>>> On Tue, Apr 18, 2017 at 11:09 AM, Evgenii Zhuravlev [via Apache Ignite
>>> Users] <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=12042&i=0>> wrote:
>>>
>>>> Which version of ignite do you use?
>>>>
>>>> It looks like nodes discovered each other via 47500+ ports, but they
>>>> can't communicate through 47100+ ports. Did you opened these ports? Are you
>>>> sure that all nodes was used in your application?
>>>>
>>>> Also, it would be helpful if you provided full logs, it's nearly
>>>> impossible to understand what really happens on the nodes without them.
>>>>
>>>> 2017-04-18 18:39 GMT+03:00 ignite_user2016 <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=12038&i=0>>:
>>>>
>>>>> Yes, I see this logs in both the host.
>>>>>
>>>>> The log entries I have provided, here is my config file -
>>>>>
>>>>> <beans xmlns="http://www.springframework.org/schema/beans"
>>>>>        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>        xmlns:util="http://www.springframework.org/schema/util"
>>>>>        xsi:schemaLocation="
>>>>>         http://www.springframework.org/schema/beans
>>>>>         http://www.springframework.org/schema/beans/spring-beans.xsd
>>>>>         http://www.springframework.org/schema/util
>>>>>         http://www.springframework.org/schema/util/spring-util.xsd">
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>     <bean id="ignite.cfg" class="org.apache.ignite.confi
>>>>> guration.IgniteConfiguration">
>>>>>
>>>>>         <property name="gridName" value="WebGrid"/>
>>>>>
>>>>>         <!-- Set to true to enable distributed class loading for
>>>>> examples, default is false. -->
>>>>>         <property name="peerClassLoadingEnabled" value="true"/>
>>>>>         <!-- Enable client mode. -->
>>>>>         <property name="clientMode" value="false"/>
>>>>>
>>>>>         <property name="gridLogger">
>>>>>             <bean class="org.apache.ignite.logger.jcl.JclLogger">
>>>>>                 <constructor-arg type="org.apache.commons.loggi
>>>>> ng.Log">
>>>>>                     <bean class="org.apache.commons.logg
>>>>> ing.impl.Log4JLogger">
>>>>>                         <constructor-arg type="java.lang.String"
>>>>> value="config/log4j.xml"/>
>>>>>                     </bean>
>>>>>                 </constructor-arg>
>>>>>             </bean>
>>>>>         </property>
>>>>>
>>>>>         <!-- Enable task execution events for examples. -->
>>>>>         <property name="includeEventTypes">
>>>>>             <list>
>>>>>                 <util:constant static-field="org.apache.ignit
>>>>> e.events.EventType.EVT_CLIENT_NODE_RECONNECTED"/>
>>>>>             </list>
>>>>>         </property>
>>>>>
>>>>>
>>>>>
>>>>>         <property name="cacheConfiguration">
>>>>>             <list>
>>>>>                 <!-- Partitioned cache example configuration (Atomic
>>>>> mode). -->
>>>>>                 <bean class="org.apache.ignite.confi
>>>>> guration.CacheConfiguration">
>>>>>                     <property name="name" value="session-cache"/>
>>>>>                     <!-- Cache mode. -->
>>>>>                     <property name="cacheMode" value="REPLICATED"/>
>>>>>                     <property name="atomicityMode" value="ATOMIC"/>
>>>>>                     <property name="backups" value="0"/>
>>>>>                     <property name="startSize" value="#{100 * 1024
*
>>>>> 1024}"/>
>>>>>                 </bean>
>>>>>             </list>
>>>>>         </property>
>>>>>
>>>>>
>>>>>
>>>>>         <property name="discoverySpi">
>>>>>             <bean class="org.apache.ignite.spi.d
>>>>> iscovery.tcp.TcpDiscoverySpi">
>>>>>                 <property name="ipFinder">
>>>>>                     <!--
>>>>>                         Ignite provides several options for automatic
>>>>> discovery that can be used
>>>>>                         instead os static IP based discovery. For
>>>>> information on all options refer
>>>>>                         to our documentation:
>>>>> http://apacheignite.readme.io/docs/cluster-config
>>>>>                     -->
>>>>>                     <!-- Uncomment static IP finder to enable
>>>>> static-based discovery of initial nodes. -->
>>>>>                     <bean class="org.apache.ignite.spi.d
>>>>> iscovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>>>>                         <!--<bean class="org.apache.ignite.spi.d
>>>>> iscovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->
>>>>>                         <property name="addresses">
>>>>>                             <list>
>>>>>                                 <!-- In distributed environment,
>>>>> replace with actual host IP address. -->
>>>>>                                 <value>HOST_IP1:47500..47509</value>
>>>>>                                 <value>HOST_IP2:47500..47509</value>
>>>>>                             </list>
>>>>>                         </property>
>>>>>                     </bean>
>>>>>                 </property>
>>>>>             </bean>
>>>>>         </property>
>>>>>
>>>>>     </bean>
>>>>> </beans>
>>>>>
>>>>> On Tue, Apr 18, 2017 at 5:46 AM, Evgenii Zhuravlev [via Apache Ignite
>>>>> Users] <[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=12037&i=0>>
wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Could you check that destination node accessible from current node
>>>>>> via one of this addresses(remote_host/remote_host:47102,
>>>>>> /0:0:0:0:0:0:0:1%lo:47102, /127.0.0.1:47102?)
>>>>>>
>>>>>> Do you have problems only with one node, or you have same messages
>>>>>> about another nodes and on another nodes?
>>>>>>
>>>>>> Please provide logs from another nodes and your config file.
>>>>>>
>>>>>> 2017-04-17 23:00 GMT+03:00 ignite_user2016 <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=12025&i=0>>:
>>>>>>
>>>>>>> I see the frequent disconnect errors in the logs -
>>>>>>>
>>>>>>> [10:34:03,028][WARN ][tcp-comm-worker-#1%WebGrid%]
>>>>>>> [TcpCommunicationSpi]
>>>>>>> Failed to connect to a remote node (make sure that destination
node
>>>>>>> is alive
>>>>>>> and operating system firewall is disabled on local and remote
hosts)
>>>>>>> [addrs=[remote_host/remote_host:47102, /0:0:0:0:0:0:0:1%lo:47102,
>>>>>>> /127.0.0.1:47102]]
>>>>>>> [11:58:06,634][WARN ][grid-nio-worker-2-#42%WebGri
>>>>>>> d%][TcpCommunicationSpi]
>>>>>>> Communication SPI Session write timed out (consider increasing
>>>>>>> 'socketWriteTimeout' configuration property)
>>>>>>> [remoteAddr=/remote_host_ip:38104, writeTimeout=2000]
>>>>>>> [12:04:04,124][WARN ][tcp-comm-worker-#1%WebGrid%]
>>>>>>> [TcpCommunicationSpi]
>>>>>>> Connect timed out (consider increasing 'failureDetectionTimeout'
>>>>>>> configuration property) [addr=/0:0:0:0:0:0:0:1%lo:47102,
>>>>>>> failureDetectionTimeout=10000]
>>>>>>> [12:04:04,124][WARN ][tcp-comm-worker-#1%WebGrid%]
>>>>>>> [TcpCommunicationSpi]
>>>>>>> Connect timed out (consider increasing 'failureDetectionTimeout'
>>>>>>> configuration property) [addr=/127.0.0.1:47102,
>>>>>>> failureDetectionTimeout=10000]
>>>>>>> [12:04:04,124][WARN ][tcp-comm-worker-#1%WebGrid%]
>>>>>>> [TcpCommunicationSpi]
>>>>>>> Failed to connect to a remote node (make sure that destination
node
>>>>>>> is alive
>>>>>>> and operating system firewall is disabled on local and remote
hosts)
>>>>>>> [addrs=[remote_host/remote_host:47102, /0:0:0:0:0:0:0:1%lo:47102,
>>>>>>> /127.0.0.1:47102]]
>>>>>>>
>>>>>>> With this errors in the logs, I see our system is operational
and
>>>>>>> does not
>>>>>>> have any failures so are these errors showing false alarms ?
>>>>>>>
>>>>>>>
>>>>>>> I am wondering what would be the idle value for socketWriteTimeOut
>>>>>>> and
>>>>>>> failureDetectionCount ? would it have any performance impact
on it ?
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Rishi
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context: http://apache-ignite-users.705
>>>>>>> 18.x6.nabble.com/Ignite-errors-in-log-tp12007.html
>>>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>>>> discussion below:
>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-errors
>>>>>> -in-log-tp12007p12025.html
>>>>>> To start a new topic under Apache Ignite Users, email [hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=12037&i=1>
>>>>>> To unsubscribe from Apache Ignite Users, click here.
>>>>>> NAML
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rishi Yagnik
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Re: Ignite errors in log
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Ignite-errors-in-log-tp12007p12037.html>
>>>>>
>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-errors
>>>> -in-log-tp12007p12038.html
>>>> To start a new topic under Apache Ignite Users, email [hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=12042&i=1>
>>>> To unsubscribe from Apache Ignite Users, click here.
>>>> NAML
>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>>
>>>
>>> --
>>> Rishi Yagnik
>>>
>>> ------------------------------
>>> View this message in context: Re: Ignite errors in log
>>> <http://apache-ignite-users.70518.x6.nabble.com/Ignite-errors-in-log-tp12007p12042.html>
>>> Sent from the Apache Ignite Users mailing list archive
>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>
>>
>>
>>
>> --
>> Rishi Yagnik
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
> Regards, Andrew.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-
> errors-in-log-tp12007p12044.html
> To start a new topic under Apache Ignite Users, email
> ml-node+s70518n1h85@n6.nabble.com
> To unsubscribe from Apache Ignite Users, click here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=cmlzaGl5YWduaWtAZ21haWwuY29tfDF8MTMwNTI4OTg1Mw==>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>



-- 
Rishi Yagnik




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-errors-in-log-tp12007p12049.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Mime
View raw message