Return-Path: X-Original-To: apmail-karaf-user-archive@minotaur.apache.org Delivered-To: apmail-karaf-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EE84A185A0 for ; Wed, 6 Jan 2016 16:15:38 +0000 (UTC) Received: (qmail 48290 invoked by uid 500); 6 Jan 2016 16:15:38 -0000 Delivered-To: apmail-karaf-user-archive@karaf.apache.org Received: (qmail 48257 invoked by uid 500); 6 Jan 2016 16:15:38 -0000 Mailing-List: contact user-help@karaf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@karaf.apache.org Delivered-To: mailing list user@karaf.apache.org Received: (qmail 48210 invoked by uid 99); 6 Jan 2016 16:15:38 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Jan 2016 16:15:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 2E2D8C1561 for ; Wed, 6 Jan 2016 16:15:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.001 X-Spam-Level: * X-Spam-Status: No, score=1.001 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id RZmjQi61eVLI for ; Wed, 6 Jan 2016 16:15:25 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id CB7A8201BC for ; Wed, 6 Jan 2016 16:15:24 +0000 (UTC) Received: from mfilter28-d.gandi.net (mfilter28-d.gandi.net [217.70.178.159]) by relay4-d.mail.gandi.net (Postfix) with ESMTP id 23937172097 for ; Wed, 6 Jan 2016 17:15:23 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mfilter28-d.gandi.net Received: from relay4-d.mail.gandi.net ([IPv6:::ffff:217.70.183.196]) by mfilter28-d.gandi.net (mfilter28-d.gandi.net [::ffff:10.0.15.180]) (amavisd-new, port 10024) with ESMTP id 2zYQ44bD6p9W for ; Wed, 6 Jan 2016 17:15:20 +0100 (CET) X-Originating-IP: 82.238.224.4 Received: from [192.168.134.10] (bre91-1-82-238-224-4.fbx.proxad.net [82.238.224.4]) (Authenticated sender: jb@nanthrax.net) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id A9E8B1720CE for ; Wed, 6 Jan 2016 17:15:19 +0100 (CET) Subject: Re: Cellar clustering issue To: user@karaf.apache.org References: <443BCE38E921434394B45FF0813FA5785053E4DB@MSGEXSIL1119.ent.wfb.bank.corp> <568BCB49.6020004@nanthrax.net> <443BCE38E921434394B45FF0813FA57850561618@MSGEXSIL1119.ent.wfb.bank.corp> <568BE0B4.4010409@nanthrax.net> <443BCE38E921434394B45FF0813FA5785056213D@MSGEXSIL1119.ent.wfb.bank.corp> <568D1C18.1020102@nanthrax.net> <443BCE38E921434394B45FF0813FA57850562184@MSGEXSIL1119.ent.wfb.bank.corp> <568D1E86.1010800@nanthrax.net> <443BCE38E921434394B45FF0813FA578505621B7@MSGEXSIL1119.ent.wfb.bank.corp> <568D1FB5.9000005@nanthrax.net> <443BCE38E921434394B45FF0813FA57850562275@MSGEXSIL1119.ent.wfb.bank.corp> <568D27F9.5010101@nanthrax.net> <443BCE38E921434394B45FF0813FA578505622FB@MSGEXSIL1119.ent.wfb.bank.corp> <568D2A9A.8070908@nanthrax.net> <443BCE38E921434394B45FF0813FA57850562341@MSGEXSIL1119.ent.wfb.bank.corp> <568D2D26.5000701@nanthrax.net> <443BCE38E921434394B45FF0813FA57850562448@MSGEXSIL1119.ent.wfb.bank.corp> From: =?UTF-8?Q?Jean-Baptiste_Onofr=c3=a9?= Message-ID: <568D3D96.9020905@nanthrax.net> Date: Wed, 6 Jan 2016 17:15:18 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <443BCE38E921434394B45FF0813FA57850562448@MSGEXSIL1119.ent.wfb.bank.corp> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cool, finally you got it ;) Regards JB On 01/06/2016 05:11 PM, barry.barnett@wellsfargo.com wrote: > It works!!!!!! > > Regards, > > Barry > > > -----Original Message----- > From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] > Sent: Wednesday, January 06, 2016 10:05 AM > To: user@karaf.apache.org > Subject: Re: Cellar clustering issue > > Both on each node > > On 01/06/2016 03:55 PM, barry.barnett@wellsfargo.com wrote: >> Just put one member, the remote one? Not the local? Or do I put both the box1 and box2 as members? >> >> Regards, >> >> Barry >> >> >> -----Original Message----- >> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >> Sent: Wednesday, January 06, 2016 9:54 AM >> To: user@karaf.apache.org >> Subject: Re: Cellar clustering issue >> >> OK, so, now, disable multicast, and enable tcp-ip with member containing IP of the different host. >> >> Regards >> JB >> >> On 01/06/2016 03:49 PM, barry.barnett@wellsfargo.com wrote: >>> Yes, still doesn't see the remote node from either box. >>> >>> Regards, >>> >>> Barry >>> >>> >>> -----Original Message----- >>> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >>> Sent: Wednesday, January 06, 2016 9:43 AM >>> To: user@karaf.apache.org >>> Subject: Re: Cellar clustering issue >>> >>> Did you do cluster:node-list to check the nodes that the cluster is able to see ? >>> >>> On 01/06/2016 03:39 PM, barry.barnett@wellsfargo.com wrote: >>>> Ok, so I try to do a cluster:group-set dev box2, from box1... >>>> >>>> cluster:group-set dev box2:5701 >>>> Cluster node box2:5701 doesn't exist >>>> >>>> 06 Jan 2016 08:33:51,141 | DEBUG | Thread-213 | LoggingCommandSessionListener | 21 - org.apache.karaf.shell.console - 2.4.3 | Command: 'cluster:group-set dev box2' returned 'null' >>>> >>>> >>>> >>>> Regards, >>>> >>>> Barry >>>> >>>> >>>> -----Original Message----- >>>> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >>>> Sent: Wednesday, January 06, 2016 9:08 AM >>>> To: user@karaf.apache.org >>>> Subject: Re: Cellar clustering issue >>>> >>>> Please, can you try: >>>> >>>> 1. disable tcp-ip and use multicast >>>> 2. disable interfaces >>>> 3. send the debug log message on each box 4. result of >>>> cluster:node-list >>>> >>>> Thanks, >>>> Regards >>>> JB >>>> >>>> On 01/06/2016 03:05 PM, barry.barnett@wellsfargo.com wrote: >>>>> In the tcpip stanza, I only have member for box1 and box2. I don't specify interface. Should I also include interface there? With just the members, its still not picking up the remote node from either box. >>>>> >>>>> Regards, >>>>> >>>>> Barry >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >>>>> Sent: Wednesday, January 06, 2016 9:03 AM >>>>> To: user@karaf.apache.org >>>>> Subject: Re: Cellar clustering issue >>>>> >>>>> If you mean inside , yes correct. >>>>> >>>>> or in can help (depending of your network configuration). >>>>> >>>>> Regards >>>>> JB >>>>> >>>>> >>>>> On 01/06/2016 02:58 PM, barry.barnett@wellsfargo.com wrote: >>>>>> So to connect one node to another remote node, it is not necessary to specify interface? Just set to false and allow to bind on all? >>>>>> >>>>>> Regards, >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >>>>>> Sent: Wednesday, January 06, 2016 8:52 AM >>>>>> To: user@karaf.apache.org >>>>>> Subject: Re: Cellar clustering issue >>>>>> >>>>>> Hi Barry, >>>>>> >>>>>> Your interface configuration is not correct. >>>>>> >>>>>> It's the network interface of your machine. >>>>>> >>>>>> By default, Hazelcast binds on all interface on your machine (0.0.0.0). >>>>>> >>>>>> You use to specify on which "local" interface you want to bind. >>>>>> For instance, you have eth0 (192.169.1.1) and eth1 (192.168.134.10) on your machine. You want to bind on eth1, so you do: >>>>>> >>>>>> >>>>>> 192.168.134.10 >>>>>> >>>>>> >>>>>> If it's not your case, I advise to disable interface in order to bind on all interfaces (0.0.0.0). >>>>>> >>>>>> As Hazelcast doesn't start, it explains why the Cellar ClusterManager service is not present. >>>>>> >>>>>> Regards >>>>>> JB >>>>>> >>>>>> On 01/06/2016 02:46 PM, barry.barnett@wellsfargo.com wrote: >>>>>>> My interface is set to: >>>>>>> >>>>>>> IPBox1 >>>>>>> IPBox2 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> 06 Jan 2016 08:34:10,339 | ERROR | FelixStartLevel | AddressPicker | 250 - com.hazelcast - 2.6.9 | Hazelcast CANNOT start on this node. No matching network interface found. >>>>>>> Interface matching must be either disabled or updated in the hazelcast.xml config file. >>>>>>> 06 Jan 2016 08:34:10,340 | ERROR | FelixStartLevel | AddressPicker | 250 - com.hazelcast - 2.6.9 | Hazelcast CANNOT start on this node. No matching network interface found. >>>>>>> Interface matching must be either disabled or updated in the hazelcast.xml config file. >>>>>>> java.lang.RuntimeException: Hazelcast CANNOT start on this node. No matching network interface found. >>>>>>> Interface matching must be either disabled or updated in the hazelcast.xml config file. >>>>>>> at com.hazelcast.impl.AddressPicker.pickAddress(AddressPicker.java:147) >>>>>>> at com.hazelcast.impl.AddressPicker.pickAddress(AddressPicker.java:51) >>>>>>> at com.hazelcast.impl.Node.(Node.java:144) >>>>>>> at com.hazelcast.impl.FactoryImpl.(FactoryImpl.java:386) >>>>>>> at com.hazelcast.impl.FactoryImpl.newHazelcastInstanceProxy(FactoryImpl.java:133) >>>>>>> at com.hazelcast.impl.FactoryImpl.newHazelcastInstanceProxy(FactoryImpl.java:119) >>>>>>> at com.hazelcast.impl.FactoryImpl.newHazelcastInstanceProxy(FactoryImpl.java:104) >>>>>>> at com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:507)[250:com.hazelcast:2.6.9] >>>>>>> at org.apache.karaf.cellar.hazelcast.factory.HazelcastServiceFactory.buildInstance(HazelcastServiceFactory.java:107)[253:org.apache.karaf.cellar.hazelcast:2.3.6] >>>>>>> at org.apache.karaf.cellar.hazelcast.factory.HazelcastServiceFactory.getInstance(HazelcastServiceFactory.java:92)[253:org.apache.karaf.cellar.hazelcast:2.3.6] >>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_45] >>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_45] >>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_45] >>>>>>> at java.lang.reflect.Method.invoke(Method.java:497)[:1.8.0_45] >>>>>>> at org.apache.aries.blueprint.utils.ReflectionUtils.invoke(ReflectionUtils.java:297)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.BeanRecipe.invoke(BeanRecipe.java:958)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.BeanRecipe.getInstance(BeanRecipe.java:298)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:806)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:787)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_45] >>>>>>> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.di.RefRecipe.internalCreate(RefRecipe.java:62)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:106)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.ServiceRecipe.createService(ServiceRecipe.java:284)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.ServiceRecipe.internalGetService(ServiceRecipe.java:251)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.ServiceRecipe.internalCreate(ServiceRecipe.java:148)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.ServiceRecipe.internalCreate(ServiceRecipe.java:148)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_45] >>>>>>> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:245)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:183)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:682)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:377)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:269)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintExtender.createContain e r > ( > B >> l >> u >>> e >>> p >>>> r >>>> i >>>>> n >>>>> t >>>>>> E >>>>>> xtender.java:294)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:263)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintExtender.modifiedBundle(BlueprintExtender.java:253)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:500)[13:org.apache.aries.util:1.1.0] at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:433)[13:org.apache.aries.util:1.1.0] at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$AbstractTracked.track(BundleHookBundleTracker.java:725)[13:org.apache.aries.util:1.1.0] at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.bundleChanged(BundleHookBundleTracker.java:463)[13:org.apache.aries.util : > 1 > >> . >> >>> 1 >>> >>>> . >>>> >>>>> 0 >>>>> ] >>>>>> >>>>>> at >>>>>> org.apache.aries.util.tracker.hook.BundleHookBundleTracker$BundleEv >>>>>> e >>>>>> n >>>>>> t >>>>>> Hook.event(BundleHookBundleTracker.java:422)[13:org.apache.aries.util: >>>>>> 1.1.0] >>>>>>> at org.apache.felix.framework.util.SecureAction.invokeBundleEventHook(SecureAction.java:1127)[org.apache.felix.framework-4.4.1.jar:] at org.apache.felix.framework.util.EventDispatcher.createWhitelistFromHooks(EventDispatcher.java:696)[org.apache.felix.framework-4.4.1.jar:] >>>>>>> at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:484)[org.apache.felix.framework-4.4.1.jar:] >>>>>>> at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4429)[org.apache.felix.framework-4.4.1.jar:] >>>>>>> at org.apache.felix.framework.Felix.startBundle(Felix.java:2100)[org.apache.felix.framework-4.4.1.jar:] at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1299)[org.apache.felix.framework-4.4.1.jar:] >>>>>>> at >>>>>>> org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkSt >>>>>>> a r t L evelImpl.java:304)[org.apache.felix.framework-4.4.1.jar:] >>>>>>> at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1299)[org.apache.felix.framework-4.4.1.jar:] >>>>>>> at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:304)[org.apache.felix.framework-4.4.1.jar:] >>>>>>> at java.lang.Thread.run(Thread.java:745)[:1.8.0_45] >>>>>>> 06 Jan 2016 08:34:10,349 | ERROR | FelixStartLevel | ServiceRecipe | 18 - org.apache.aries.blueprint.core - 1.4.3 | Error retrieving service from ServiceRecipe[name='.component-1'] >>>>>>> org.osgi.service.blueprint.container.ComponentDefinitionException: Error when instantiating bean hazelcast of class com.hazelcast.core.Hazelcast >>>>>>> at org.apache.aries.blueprint.container.BeanRecipe.getInstance(BeanRecipe.java:300)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:806)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:787)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_45] >>>>>>> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.di.RefRecipe.internalCreate(RefRecipe.java:62)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:106)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.ServiceRecipe.createService(ServiceRecipe.java:284)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.ServiceRecipe.internalGetService(ServiceRecipe.java:251)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at org.apache.aries.blueprint.container.ServiceRecipe.internalCreate(ServiceRecipe.java:148)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)[18:org.apache.aries.blueprint.core:1.4.3] >>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_45] at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:245)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:183)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:682)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:377)[18:org.apache.aries.blueprint.core:1.4.3] at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:269)[18:org.apache.aries.blueprint.core:1.4.3] >>> >>> a >>>> t >>>> >>>>> o >>>>> r >>>>>> g >>>>>> .apache.aries.blueprint.container.BlueprintExtender.createContain >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Barry >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >>>>>>> Sent: Tuesday, January 05, 2016 10:27 AM >>>>>>> To: user@karaf.apache.org >>>>>>> Subject: Re: Cellar clustering issue >>>>>>> >>>>>>> Can you set log level in DEBUG and send karaf.log to me ? >>>>>>> >>>>>>> It looks like bundle-hazelcast doesn't expose the ClusterManager service. >>>>>>> >>>>>>> Regards >>>>>>> JB >>>>>>> >>>>>>> On 01/05/2016 03:42 PM, barry.barnett@wellsfargo.com wrote: >>>>>>>> Ok, now I've put in the stanza for required-members. But when I do that, and have interfaces enabled, I get the following: >>>>>>>> >>>>>>>> 05 Jan 2016 09:40:20,739 | INFO | Thread-198 | ReferenceRecipe | 18 - org.apache.aries.blueprint.core - 1.4.3 | No matching service for optional OSGi service reference (objectClass=org.apache.karaf.cellar.core.ClusterManager) >>>>>>>> 05 Jan 2016 09:40:20,741 | ERROR | Thread-198 | Console | 21 - org.apache.karaf.shell.console - 2.4.3 | Exception caught while executing command >>>>>>>> org.osgi.service.blueprint.container.ServiceUnavailableException: No matching service for optional OSGi service reference: (objectClass=org.apache.karaf.cellar.core.ClusterManager) >>>>>>>> at org.apache.aries.blueprint.container.ReferenceRecipe.getService(ReferenceRecipe.java:236) >>>>>>>> at org.apache.aries.blueprint.container.ReferenceRecipe.access$000(ReferenceRecipe.java:55) >>>>>>>> at org.apache.aries.blueprint.container.ReferenceRecipe$ServiceDispatcher.call(ReferenceRecipe.java:298) >>>>>>>> at Proxy2882e1b3_fe00_4c30_818e_5ad671ebc492.listNodes(Unknown Source) >>>>>>>> at org.apache.karaf.cellar.shell.NodesListCommand.doExecute(NodesListCommand.java:29) >>>>>>>> at org.apache.karaf.shell.console.OsgiCommandSupport.execute(OsgiCommandSupport.java:38) >>>>>>>> at org.apache.felix.gogo.commands.basic.AbstractCommand.execute(AbstractCommand.java:35) >>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_45] >>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_45] >>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_45] >>>>>>>> at java.lang.reflect.Method.invoke(Method.java:497)[:1.8.0_45] >>>>>>>> at org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:54) >>>>>>>> at org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:119) >>>>>>>> at org.apache.karaf.shell.console.commands.$BlueprintCommand1144753357.execute(Unknown Source)[21:org.apache.karaf.shell.console:2.4.3] >>>>>>>> at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:78)[21:org.apache.karaf.shell.console:2.4.3] >>>>>>>> at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:477)[21:org.apache.karaf.shell.console:2.4.3] >>>>>>>> at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:403)[21:org.apache.karaf.shell.console:2.4.3] >>>>>>>> at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)[21:org.apache.karaf.shell.console:2.4.3] >>>>>>>> at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183)[21:org.apache.karaf.shell.console:2.4.3] >>>>>>>> at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120)[21:org.apache.karaf.shell.console:2.4.3] >>>>>>>> at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:92) >>>>>>>> at org.apache.karaf.shell.console.jline.Console.run(Console.java:195) >>>>>>>> at org.apache.karaf.shell.ssh.ShellFactoryImpl$ShellImpl$1.runConsole(ShellFactoryImpl.java:167)[36:org.apache.karaf.shell.ssh:2.4.3] >>>>>>>> at org.apache.karaf.shell.ssh.ShellFactoryImpl$ShellImpl$1$1.run(ShellFactoryImpl.java:126) >>>>>>>> at java.security.AccessController.doPrivileged(Native Method)[:1.8.0_45] >>>>>>>> at org.apache.karaf.jaas.modules.JaasHelper.doAs(JaasHelper.java:47)[20:org.apache.karaf.jaas.modules:2.4.3] >>>>>>>> at >>>>>>>> org.apache.karaf.shell.ssh.ShellFactoryImpl$ShellImpl$1.run(Shell >>>>>>>> F a c t o ryImpl.java:124)[36:org.apache.karaf.shell.ssh:2.4.3] >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Barry >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >>>>>>>> Sent: Tuesday, January 05, 2016 8:55 AM >>>>>>>> To: user@karaf.apache.org >>>>>>>> Subject: Re: Cellar clustering issue >>>>>>>> >>>>>>>> As I'm not able to reproduce your issue, it's not easy to figure it out. >>>>>>>> >>>>>>>> Clearly, the problem is that the nodes don't see each other, and I suspect we're missing something obvious on the network configuration. So yes, tweaking the tcp-ip configuration can help. >>>>>>>> >>>>>>>> The only weird thing for me is the fact that you have a Cellar bundle in failed state. Is it still the case ? >>>>>>>> >>>>>>>> Regards >>>>>>>> JB >>>>>>>> >>>>>>>> On 01/05/2016 02:50 PM, barry.barnett@wellsfargo.com wrote: >>>>>>>>> Should I try the following? >>>>>>>>> >>>>>>>>> >>>>>>>>> IPofBox1 >>>>>>>>> IPofBox1 >>>>>>>>> IPofBox1,IPofBox2 >>>>>>>>> >>>>>>>>> >>>>>>>>> I currently only use: >>>>>>>>> >>>>>>>>> IPofBox1 >>>>>>>>> IPofBox2 >>>>>>>>> >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Barry >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Jean-Baptiste Onofr� [mailto:jb@nanthrax.net] >>>>>>>>> Sent: Monday, January 04, 2016 11:09 AM >>>>>>>>> To: user@karaf.apache.org >>>>>>>>> Subject: Re: Cellar clustering issue >>>>>>>>> >>>>>>>>> Do you mind to provide a link to the karaf log ? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Regards >>>>>>>>> JB >>>>>>>>> >>>>>>>>> On 01/04/2016 04:59 PM, barry.barnett@wellsfargo.com wrote: >>>>>>>>>> Iptables is not enabled on the Linux boxes. >>>>>>>>>> >>>>>>>>>> Config on hazelcast.xml for Box1. Box2 is a mirror image basically: >>>>>>>>>> >>>>>>>>>> >>>>>>>>> xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-2.5.xsd" >>>>>>>>>> xmlns="http://www.hazelcast.com/schema/config" >>>>>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> >>>>>>>>>> >>>>>>>>>> dev >>>>>>>>>> pass >>>>>>>>>> >>>>>>>>>> http://localhost:8080/mancenter >>>>>>>>>> >>>>>>>>>> 5701 >>>>>>>>>> >>>>>>>>>> 0 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 224.2.2.3 >>>>>>>>>> 54327 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> IPforBox1:5701 >>>>>>>>>> IPforBox2:5701 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> my-access-key >>>>>>>>>> my-secret-key >>>>>>>>>> >>>>>>>>>> us-west-1 >>>>>>>>>> >>>>>>>>>> ec2.amazonaws.com >>>>>>>>>> >>>>>>>>>> hazelcast-sg >>>>>>>>>> type >>>>>>>>>> hz-nodes >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> IPforBox1 >>>>>>>>>> IPforBox2 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> PBEWithMD5AndDES >>>>>>>>>> >>>>>>>>>> thesalt >>>>>>>>>> >>>>>>>>>> thepass >>>>>>>>>> >>>>>>>>>> 19 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> RSA/NONE/PKCS1PADDING >>>>>>>>>> >>>>>>>>>> thekeypass >>>>>>>>>> >>>>>>>>>> local >>>>>>>>>> >>>>>>>>>> JKS >>>>>>>>>> >>>>>>>>>> thestorepass >>>>>>>>>> >>>>>>>>>> keystore >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 16 >>>>>>>>>> 64 >>>>>>>>>> 60 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 0 >>>>>>>>>> >>>>>>>>>> default >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 1 >>>>>>>>>> >>>>>>>>>> 0 >>>>>>>>>> >>>>>>>>>> 0 >>>>>>>>>> >>>>>>>>>> NONE >>>>>>>>>> >>>>>>>>>> 0 >>>>>>>>>> >>>>>>>>>> 25 >>>>>>>>>> >>>>>>>>>> hz.ADD_NEW_ENTRY >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>