cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangeetha Hariharan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-4617) [UI] Xenserver 6.1/ 6.2 - Add host succeeds but gets into Alert state due to "Unable to create local link network". It remains in "Alert" state for a while and then gets to "UP" state.
Date Tue, 17 Sep 2013 18:48:55 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769804#comment-13769804
] 

Sangeetha Hariharan commented on CLOUDSTACK-4617:
-------------------------------------------------

Tested with latest build from 4.2-forward branch.

When first host gets added as part of zone wizard creation ,we see the host gets added with
no exception seen in management server logs. Also when addHost() API is returned , state of
the host is in "Up" state. So we do not see the primary storage failure that was noted before.

apilog.log.2013-09-16:2013-09-16 13:44:03,616 INFO  [cloud.api.ApiServer] (catalina-exec-16:null)
(userId=2 accountId=2 sessionId=D5A90488C21BFE3E1F3C300BADF37CA3) 10.216.51.146 -- POST command=addHost&response=json&sessionkey=njrZG%2BQIhWlkPR7gjm2XrC%2Fhvsc%3D
200 { "addhostresponse" : { "count":1 ,"host" : [  {"id":"32ef72c4-a92a-4293-b5e9-59e10e884f61","name":"xenserver-sangeetha1","state":"Up","type":"Routing","ipaddress":"10.223.210.21","zoneid":"7ba002d6-d608-478d-98e4-ce71787a97d5","zonename":"zone1","podid":"7f5861d9-cb02-421f-a503-225c392201d7","podname":"pod1","version":"4.2.0","hypervisor":"XenServer","cpunumber":4,"cpuspeed":2261,"cpuallocated":"0%","cpuwithoverprovisioning":"9044.0","memorytotal":3701658240,"memoryallocated":0,"capabilities":"xen-3.0-x86_64
, xen-3.0-x86_32p , hvm-3.0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64","lastpinged":"1970-01-16T06:10:35-0800","managementserverid":161197867246747,"clusterid":"798026b6-2090-4e7a-a4ef-795971321e16","clustername":"cluster1","clustertype":"CloudManaged","islocalstorageactive":false,"created":"2013-09-16T13:43:45-0700","events":"AgentDisconnected;
Remove; StartAgentRebalance; HostDown; Ping; PingTimeout; ManagementServerDown; AgentConnected;
ShutdownRequested","resourcestate":"Enabled","hypervisorversion":"6.1.0","hahost":false} ]
} }


When I try to add second host to a cluster (which already has 1 host) , we see the host addition
succeeds. But we see the host in "Alert" state then transitioning to "Connecting" state and
after a while gets to "Up" state.

apilog.log:2013-09-17 11:15:48,937 INFO  [cloud.api.ApiServer] (catalina-exec-1:null) (userId=2
accountId=2 sessionId=3973A205C2F1BCD4FC41F46D9BBCEB66) 10.215.3.9 -- POST command=addHost&response=json&sessionkey=4Xt2OZIdOBK4kvtp%2BRWMau%2B%2Bj0I%3D
200 { "addhostresponse" : { "count":1 ,"host" : [  {"id":"769e9041-3fc6-4a37-91af-93e7e2c2139d","name":"Rack2Host16.lab.vmops.com","state":"Alert","type":"Routing","ipaddress":"10.223.50.195","zoneid":"a465e027-8a96-42c1-87f4-f3bc2efd1dce","zonename":"zone2","podid":"0e7e9cb3-6fa8-4101-890d-4520fc34de6b","podname":"pod2","version":"4.2.0","hypervisor":"XenServer","cpunumber":4,"cpuspeed":2261,"cpuallocated":"0%","cpuwithoverprovisioning":"9044.0","memorytotal":16189931520,"memoryallocated":0,"capabilities":"xen-3.0-x86_64
, xen-3.0-x86_32p , hvm-3.0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64","lastpinged":"1969-12-31T16:00:00-0800","clusterid":"914a29ce-9384-4715-b758-e1ee04bb7c3c","clustername":"cluster2","clustertype":"CloudManaged","islocalstorageactive":false,"created":"2013-09-17T11:15:48-0700","events":"ManagementServerDown;
Remove; AgentConnected; AgentDisconnected; Ping; ShutdownRequested","resourcestate":"Enabled","hypervisorversion":"6.1.0","hahost":false}
] } }


                
> [UI] Xenserver 6.1/ 6.2 - Add host succeeds but gets into Alert state due to "Unable
to create local link network". It remains in "Alert" state for a while and then gets to "UP"
state.
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-4617
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4617
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.0, 4.2.1
>         Environment: Build from 4.2-forward
>            Reporter: Sangeetha Hariharan
>            Assignee: Devdeep Singh
>            Priority: Critical
>             Fix For: 4.2.1
>
>         Attachments: hostdown.rar
>
>
> Xenserver 6.1 - Add host succeeds but gets into Alert state. It remains in "Alert" state
for a while and then gets to "UP" state.
> Steps to reproduce the problem:
> In my case I already had 1 zone.
> Create another advanced zone with 1 Xenserver host.
> As part of zone creation wizard , provide all the values for zone creation including
primary and secondary storage details.
> Primary storage creation fails with error - "Failed to delete storage pool on host"
>  Following exception seen in management server logs:
> 2013-09-05 11:51:25,894 DEBUG [cloud.api.ApiServlet] (catalina-exec-20:null) ===START===
 10.215.3.9 -- GET  comman
> d=createStoragePool&zoneid=b34c3dc3-27c3-4f9a-9173-c74b04a4acab&podId=fde823c6-f763-480e-a5bf-c96f4e6e43fa&clusteri
> d=3658d76d-fa9a-41c1-95a9-0c1688c2707c&name=ps1&scope=cluster&url=nfs%3A%2F%2F10.223.110.232%2Fexport%2Fhome%2Fsang
> eetha%2F307%2Fzone2-primary&response=json&sessionkey=e4LjGKV4k%2FmbX%2BczFre8RM8zhXI%3D&_=1378408027682
> 2013-09-05 11:51:25,975 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl]
(catalina-exec-20:null)
>  createPool Params @ scheme - nfs storageHost - 10.223.110.232 hostPath - /export/home/sangeetha/307/zone2-primary
> port - -1
> 2013-09-05 11:51:26,125 DEBUG [cloud.storage.StorageManagerImpl] (catalina-exec-20:null)
Failed to add data store
> com.cloud.utils.exception.CloudRuntimeException: No host up to associate a storage pool
with in cluster 2
>         at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:371)
>         at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:749)
>         at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:177)
>         at org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:168)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>         at com.cloud.api.ApiServer.queueCommand(ApiServer.java:514)
>         at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
>         at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
>         at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>         at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>         at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>         at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>         at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>         at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>         at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>         at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>         at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
>         at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
>         at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-09-05 11:51:26,173 INFO  [cloud.api.ApiServer] (catalina-exec-20:null) Failed to
delete storage pool on host
> The reason for this failure was that the host was in "Alert" state , when the primary
storage was being added.
> Following exception seen in management server log:
> 2013-09-05 11:51:25,038 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-100:null)
Lowest available Vif device number: 0 for VM: Control domain on host: Rack3Host3.lab.vmops.com
> 2013-09-05 11:51:25,462 WARN  [xen.resource.CitrixResourceBase] (DirectAgent-100:null)
Unable to create local link network
> The server failed to handle your request, due to an internal error.  The given message
may give details useful for debugging the problem.
>         at com.xensource.xenapi.Types.checkResponse(Types.java:1694)
>         at com.xensource.xenapi.Connection.dispatch(Connection.java:368)
>         at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909)
>         at com.xensource.xenapi.VIF.plug(VIF.java:846)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4871)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3391)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
>         at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
>         at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:104)
>         at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-09-05 11:51:25,463 WARN  [agent.manager.DirectAgentAttache] (DirectAgent-100:null)
Seq 6-975634439: Exception Caught while executing command
> com.cloud.utils.exception.CloudRuntimeException: Unable to create local link network
due to The server failed to handle your request, due to an internal error.  The given message
may give details useful for debugging the problem.
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4885)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3391)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
>         at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> addHost command in this case does not error out. Instead sends host details with "state"
indicating it is in "Alert" state.
> 2013-09-05 16:57:03,090 INFO  [cloud.api.ApiServer] (catalina-exec-15:null) (userId=2
accountId=2 sessionId=72F8ED
> D7B6A93CF324AD1991DC69D405) 10.215.3.9 -- POST command=addHost&response=json&sessionkey=cZmPlLscwjfca5vffTZMd7gFYw
> A%3D 200 { "addhostresponse" : { "count":1 ,"host" : [  {"id":"2f3a8c1b-cffd-48dd-8c9e-bd2c564784de","name":"Rack3
> Host4.lab.vmops.com","state":"Alert","type":"Routing","ipaddress":"10.223.57.4","zoneid":"b34c3dc3-27c3-4f9a-9173-
> c74b04a4acab","zonename":"zone2","podid":"fde823c6-f763-480e-a5bf-c96f4e6e43fa","podname":"pod1","version":"4.2.0"
> ,"hypervisor":"XenServer","cpunumber":4,"cpuspeed":2261,"cpuallocated":"0%","cpuwithoverprovisioning":"9044.0","me
> morytotal":16189931520,"memoryallocated":0,"capabilities":"xen-3.0-x86_64 , xen-3.0-x86_32p
, hvm-3.0-x86_32 , hvm
> -3.0-x86_32p , hvm-3.0-x86_64","lastpinged":"1969-12-31T16:00:00-0800","managementserverid":6612941014073,"cluster
> id":"3658d76d-fa9a-41c1-95a9-0c1688c2707c","clustername":"cluster2","clustertype":"CloudManaged","islocalstorageac
> tive":false,"created":"2013-09-05T16:57:03-0700","events":"ManagementServerDown; Remove;
ShutdownRequested; AgentC
> onnected; AgentDisconnected; Ping","resourcestate":"Enabled","hypervisorversion":"6.1.0","hahost":false}
] } }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message