cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kirk Kosinski <kirkkosin...@gmail.com>
Subject Re: Unable to add more XCP host in cluster
Date Tue, 09 Oct 2012 02:38:09 GMT
Did you check the storage configuration to see why the new host cannot
access it?  There may be some additional details on the error in
/var/log/xensource.log and SMlog.

Regards,
Kirk

On 10/08/2012 06:58 PM, Hieu Le wrote:
> Hi everyone,
> 
> I have some troubles while adding more XCP host in cluster. I have 2
> running hosts which are homogeneous and another host that is inhomogeneous.
> Now I want to add this inhomogeneous host to CS in the same cluster with 2
> running hosts but it is always fail. The step I follow is:
> 
>    - Force the inhomogeneous to join existing pool with xe command (xe
>    pool-join --force=true --master-address=....)
>    - Add new host in the same cluster with 2 existing hosts via CS UI.
>    - Getting the error.
> 
> The error log I got here is:
> 
> 2012-10-09 09:19:04,334 DEBUG [cloud.storage.StorageManagerImpl]
> (catalina-exec-2:null) Adding pool XenPrimary to  host 14
> 2012-10-09 09:19:04,336 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Sending  { Cmd , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePoolC$
> 2012-10-09 09:19:04,337 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Executing:  { Cmd , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePo$
> 2012-10-09 09:19:04,337 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-20:null) Seq 14-976879620: Executing request
> 2012-10-09 09:19:04,381 WARN  [xen.resource.CitrixResourceBase]
> (DirectAgent-20:null) ModifyStoragePoolCommand add XenAPIException:Can not
> see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732$
> com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
> b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> host:1949af01-e15e-467c-8d27-42678a5f826c
>         at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
>         at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
>         at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
>         at
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
>         at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2012-10-09 09:19:04,382 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-20:null) Seq 14-976879620: Response Received:
> 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> (DirectAgent-20:null) Seq 14-976879620: Processing:  { Ans: , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 10, [{"Answer":{"result":$
> 2012-10-09 09:19:04,382 WARN  [cloud.vm.VirtualMachineManagerImpl]
> (DirectAgent-20:null) Cleanup failed due to ModifyStoragePoolCommand add
> XenAPIException:Can not see storage pool: b43e6f96-d0$
> 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Received:  { Ans: , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 10, { Answer } }
> 2012-10-09 09:19:04,383 WARN  [cloud.vm.VirtualMachineManagerImpl]
> (catalina-exec-2:null) Cleanup failed due to ModifyStoragePoolCommand add
> XenAPIException:Can not see storage pool: b43e6f96-d$
> 2012-10-09 09:19:04,383 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Details from executing class
> com.cloud.agent.api.ModifyStoragePoolCommand: ModifyStoragePoolCommand add
> Xen$
> 2012-10-09 09:19:04,461 WARN  [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Monitor StoragePoolMonitor says there is an error in
> the connect process for 14 due to Unable to connect to$
> 2012-10-09 09:19:04,461 INFO  [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Host 14 is disconnecting with event AgentDisconnected
> 2012-10-09 09:19:04,462 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) The next status of agent 14is Alert, current status
> is Connecting
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Deregistering link for 14 with state Alert
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Remove Agent : 14
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.DirectAgentAttache]
> (catalina-exec-2:null) Processing disconnect 14
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$4$
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.vmware.VmwareManagerImpl$$EnhancerByCGLIB$$8d05fd47
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$4b7c62a0
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.security.SecurityGroupListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.listener.StoragePoolMonitor
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.secondary.SecondaryStorageListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$f53c41bc
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$65dcc41c
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.download.DownloadListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.upload.UploadListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.SshKeysDistriMonitor
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl$$EnhancerByCGLI$
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.capacity.StorageCapacityListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.capacity.ComputeCapacityListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.consoleproxy.ConsoleProxyListener
> 2012-10-09 09:19:04,464 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2012-10-09 09:19:04,464 DEBUG [cloud.network.NetworkUsageManagerImpl]
> (catalina-exec-2:null) Disconnected called on 14 with status Alert
> 2012-10-09 09:19:04,464 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
> id = 14, name = node1.viettelrd.vn]
> 2012-10-09 09:19:04,532 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Agent status update: [id = 14; name = node1.viettelrd.vn; old status =
> Connecting; event = AgentDisconnected; new status$
> 2012-10-09 09:19:04,532 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (catalina-exec-2:null) Notifying other nodes of to disconnect
> 2012-10-09 09:19:04,534 WARN  [cloud.resource.ResourceManagerImpl]
> (catalina-exec-2:null) Unable to connect due to
> com.cloud.exception.ConnectionException: Unable to connect to pool
> Pool[200|NetworkFilesystem]
>         at
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:91)
>         at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:601)
>         at
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1496)
>         at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1574)
>         at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
>         at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
>         at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
>         at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
>         at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
>         at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
>         at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>         at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>         at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>         at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>         at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
>         at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
>         at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> Caused by: com.cloud.exception.StorageUnavailableException: Resource
> [StoragePool:200] is unreachable: Unable establish connection from storage
> head to storage pool 200 due to ModifyStoragePool$
>         at
> com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1567)
>         at
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:88)
>         ... 28 more
> 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
> id = 14, name = node1.viettelrd.vn]
> 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Cannot transit agent status with event AgentDisconnected for host 14, name=
> node1.viettelrd.vn, mangement server id is 14$
> 2012-10-09 09:19:04,535 ERROR [cloud.api.ApiDispatcher]
> (catalina-exec-2:null) Exception while executing AddHostCmd:
> com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent
> status with event AgentDisconnected for host 14, mangement server id is
> 148045887322015,Unable to transition to a new state$
>         at
> com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(AgentManagerImpl.java:1428)
>         at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1588)
>         at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
>         at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
>         at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
>         at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
>         at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
>         at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
>         at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>         at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>         at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>         at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>         at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
>         at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
>         at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2012-10-09 09:19:08,143 WARN  [xen.resource.CitrixResourceBase]
> (DirectAgent-9:null) ModifyStoragePoolCommand add XenAPIException:Can not
> see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732 $
> com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
> b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> host:1949af01-e15e-467c-8d27-42678a5f826c
>         at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
>         at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
>         at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
>         at
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
>         at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 
> - When force joining the inhomogeneous host, Xen Center throws an error
> that it can not see the SR (primary storage) and can not connect to host
> via PBD.
> 
> Please support !
> 
> Best regards !
> 

Mime
View raw message