cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-9285) Cloudstack 4.8 can't connect to XEN and KVM hosts
Date Thu, 24 Mar 2016 16:45:25 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210529#comment-15210529
] 

ASF GitHub Bot commented on CLOUDSTACK-9285:
--------------------------------------------

Github user kiwiflyer commented on the pull request:

    https://github.com/apache/cloudstack/pull/1430#issuecomment-200918379
  
    @rafaelweingartner Yes, you are correct. I like your suggestion. I'll log the exception
with the failure.
    
    In terms of the persistent connections, I believe the management server pushes to the
agents. It also relies on the agent connection to determine health information for the agent
and the VMs located on each host.


> Cloudstack 4.8 can't connect to XEN and KVM hosts
> -------------------------------------------------
>
>                 Key: CLOUDSTACK-9285
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9285
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: KVM, Xen
>    Affects Versions: 4.8.0
>         Environment: CentOS 7
>            Reporter: Jens Fettig
>            Priority: Blocker
>         Attachments: agentfailure.pcap
>
>
> We used Cloudstack 4.7 in our testing environment. For future uses we updated to Cloudstack
4.8 for some tests. But after the update from 4.7 to 4.8 cloudstack has some problems to connect
to the hosts again. Here are the error logs.
> {code:title=XEN Server from the managment-server.log:|borderStyle=solid}
> 2016-02-15 11:22:03,476 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (ClusteredAgentManager
Timer:ctx-43ef0a0a) (logid:82d7b1ec) Loading directly connected host 13(xen5)
> 2016-02-15 11:22:06,204 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-aaf933f3)
(logid:d157e37d) AutoScaling Monitor is running...
> 2016-02-15 11:22:06,206 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-8e2fc084)
(logid:379074cb) VmStatsCollector is running...
> 2016-02-15 11:22:06,217 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-cc1cf960)
(logid:b6397e2e) HostStatsCollector is running...
> 2016-02-15 11:22:06,298 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-efda9c07)
(logid:5731423e) StorageCollector is running...
> 2016-02-15 11:22:06,302 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-efda9c07)
(logid:5731423e) There is no secondary storage VM for secondary storage host cdsdev-secondary
> 2016-02-15 11:22:11,018 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bc740746)
(logid:cc619bad) Begin cleanup expired async-jobs
> 2016-02-15 11:22:11,024 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bc740746)
(logid:cc619bad) End cleanup expired async-jobs
> 2016-02-15 11:22:11,055 DEBUG [c.c.h.x.r.XenServerConnectionPool] (ClusteredAgentManager
Timer:ctx-43ef0a0a) (logid:82d7b1ec) Unable to create master connection to host(192.168.0.97)
, due to The credentials given by the user are incorrect, so access has been denied, and you
have not been issued a session handle.
> 2016-02-15 11:22:11,055 DEBUG [c.c.h.Status] (ClusteredAgentManager Timer:ctx-43ef0a0a)
(logid:82d7b1ec) Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
id = 13, name = xen5]
> 2016-02-15 11:22:11,101 WARN  [c.c.a.m.ClusteredAgentManagerImpl] (ClusteredAgentManager
Timer:ctx-43ef0a0a) (logid:82d7b1ec)  can not load directly connected host 13(xen5) due to

> com.cloud.utils.exception.CloudRuntimeException: Unable to create master connection to
host(192.168.0.97) , due to The credentials given by the user are incorrect, so access has
been denied, and you have not been issued a session handle.
> 	at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool.getConnect(XenServerConnectionPool.java:163)
> 	at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.CheckXenHostInfo(CitrixResourceBase.java:523)
> 	at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.configure(CitrixResourceBase.java:827)
> 	at com.cloud.resource.DiscovererBase.reloadResource(DiscovererBase.java:158)
> 	at com.cloud.agent.manager.AgentManagerImpl.loadDirectlyConnectedHost(AgentManagerImpl.java:697)
> 	at com.cloud.agent.manager.ClusteredAgentManagerImpl.scanDirectAgentToLoad(ClusteredAgentManagerImpl.java:217)
> 	at com.cloud.agent.manager.ClusteredAgentManagerImpl.runDirectAgentScanTimerTask(ClusteredAgentManagerImpl.java:182)
> 	at com.cloud.agent.manager.ClusteredAgentManagerImpl.access$100(ClusteredAgentManagerImpl.java:96)
> 	at com.cloud.agent.manager.ClusteredAgentManagerImpl$DirectAgentScanTimerTask.runInContext(ClusteredAgentManagerImpl.java:233)
> 	at org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.runInContext(ManagedContextTimerTask.java:30)
> 	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> 	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> 	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> 	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> 	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> 	at org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:27)
> 	at java.util.TimerThread.mainLoop(Timer.java:555)
> 	at java.util.TimerThread.run(Timer.java:505)
> Caused by: The credentials given by the user are incorrect, so access has been denied,
and you have not been issued a session handle.
> 	at com.xensource.xenapi.Types.checkResponse(Types.java:322)
> 	at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
> 	at com.xensource.xenapi.Session.loginWithPassword(Session.java:587)
> 	... 19 more 
> {code}
> This seems to be the problem:
> Unable to create master connection to host(192.168.0.97) , due to The credentials given
by the user are incorrect, so access has been denied, and you have not been issued a session
handle.
> I updated the vhd-utils like in the upgrade gourde from 4.6:
> http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.7.0/upgrade/upgrade-4.6.html
> But maybe a need a new vhd-utils file for 4.8, because this one was for 4.7? 
> {code:title=KVM from the agent.log:|borderStyle=solid}
> 2016-02-15 10:46:20,971 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Agent started
> 2016-02-15 10:46:20,973 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Implementation
Version is 4.8.0
> 2016-02-15 10:46:20,974 INFO  [cloud.agent.AgentShell] (main:null) (logid:) agent.properties
found at /etc/cloudstack/agent/agent.properties
> 2016-02-15 10:46:20,977 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Defaulting
to using properties file for storage
> 2016-02-15 10:46:20,978 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Defaulting
to the constant time backoff algorithm
> 2016-02-15 10:46:20,988 INFO  [cloud.utils.LogUtils] (main:null) (logid:) log4j configuration
found at /etc/cloudstack/agent/log4j-cloud.xml
> 2016-02-15 10:46:20,999 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Preferring
IPv4 address family for agent connection
> 2016-02-15 10:46:21,095 INFO  [cloud.agent.Agent] (main:null) (logid:) id is 17
> 2016-02-15 10:46:21,116 INFO  [kvm.resource.LibvirtConnection] (main:null) (logid:) No
existing libvirtd connection found. Opening a new one
> 2016-02-15 10:46:21,373 INFO  [org.reflections.Reflections] (main:null) (logid:) Reflections
took 66 ms to scan 1 urls, producing 7 keys and 10 values
> 2016-02-15 10:46:21,514 INFO  [kvm.resource.LibvirtComputingResource] (main:null) (logid:)
No libvirt.vif.driver specified. Defaults to BridgeVifDriver.
> 2016-02-15 10:46:21,543 INFO  [cloud.agent.Agent] (main:null) (logid:) Agent [id = 17
: type = LibvirtComputingResource : zone = 1 : pod = 1 : workers = 5 : host = 192.168.0.72
: port = 8250
> 2016-02-15 10:46:21,549 INFO  [utils.nio.NioClient] (main:null) (logid:) Connecting to
192.168.0.72:8250
> 2016-02-15 10:46:21,609 ERROR [utils.nio.NioConnection] (main:null) (logid:) Unable to
initialize the threads.
> java.io.IOException: Connection closed with -1 on reading size.
>         at com.cloud.utils.nio.Link.doHandshake(Link.java:513)
>         at com.cloud.utils.nio.NioClient.init(NioClient.java:80)
>         at com.cloud.utils.nio.NioConnection.start(NioConnection.java:88)
>         at com.cloud.agent.Agent.start(Agent.java:228)
>         at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:399)
>         at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:367)
>         at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:351)
>         at com.cloud.agent.AgentShell.start(AgentShell.java:461)
>         at com.cloud.agent.AgentShell.main(AgentShell.java:496)
> 2016-02-15 10:46:21,611 INFO  [utils.exception.CSExceptionErrorCode] (main:null) (logid:)
Could not find exception: com.cloud.utils.exception.NioConnectionException in error code list
for exceptions
> 2016-02-15 10:46:21,612 ERROR [cloud.agent.AgentShell] (main:null) (logid:) Unable to
start agent:
> com.cloud.utils.exception.CloudRuntimeException: Unable to start the connection!
>         at com.cloud.agent.Agent.start(Agent.java:230)
>         at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:399)
>         at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:367)
>         at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:351)
>         at com.cloud.agent.AgentShell.start(AgentShell.java:461)
>         at com.cloud.agent.AgentShell.main(AgentShell.java:496)
> Caused by: com.cloud.utils.exception.NioConnectionException: Connection closed with -1
on reading size.
>         at com.cloud.utils.nio.NioConnection.start(NioConnection.java:94)
>         at com.cloud.agent.Agent.start(Agent.java:228)
>         ... 5 more
> Caused by: java.io.IOException: Connection closed with -1 on reading size.
>         at com.cloud.utils.nio.Link.doHandshake(Link.java:513)
>         at com.cloud.utils.nio.NioClient.init(NioClient.java:80)
>         at com.cloud.utils.nio.NioConnection.start(NioConnection.java:88)
>         ... 6 more
> {code}
> cloudstack-agent on the kvm host and the cloudstack-management on the cloudstack server
is running. Everything on version 4.8.0.
> Maybe there are some bugs or did I something wrong?
> Greetings J. Fettig



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message