cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Lamirault (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CLOUDSTACK-528) Failed to create secondary storage
Date Fri, 03 May 2013 12:36:17 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648359#comment-13648359
] 

Nicolas Lamirault edited comment on CLOUDSTACK-528 at 5/3/13 12:36 PM:
-----------------------------------------------------------------------

I add some logs :
2013-05-03 13:58:51,752 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Trying
to add a new host at nfs://xxxxxxxxx/secondarystorage in data center 1
2013-05-03 13:58:51,753 DEBUG [storage.secondary.SecondaryStorageDiscoverer] (TP-Processor18:null)
Executing: sudo mount rxxxxxx:/xxxxxxx/secondarystorage /mnt/cloudstack/7e
fdc0d 
2013-05-03 13:58:52,273 DEBUG [storage.secondary.SecondaryStorageDiscoverer] (TP-Processor18:null)
Execution is successful.
2013-05-03 13:58:52,274 DEBUG [storage.secondary.SecondaryStorageDiscoverer] (TP-Processor18:null)
Executing: sudo umount /mnt/cloudstack/7efdc0d 
2013-05-03 13:58:52,341 DEBUG [storage.secondary.SecondaryStorageDiscoverer] (TP-Processor18:null)
Execution is successful.
2013-05-03 13:58:52,363 WARN  [storage.resource.NfsSecondaryStorageResource] (TP-Processor18:null)
Wait, what's going on? eth1ip is null!!
2013-05-03 13:58:52,363 WARN  [storage.resource.NfsSecondaryStorageResource] (TP-Processor18:null)
Wait, there is no storageip in /proc/cmdline, something wrong!
2013-05-03 13:58:52,364 DEBUG [cloud.resource.ServerResourceBase] (TP-Processor18:null) Retrieving
network interface: xenbr0
2013-05-03 13:58:52,364 DEBUG [cloud.resource.ServerResourceBase] (TP-Processor18:null) Unable
to get network interface for xenbr0
2013-05-03 13:58:52,364 DEBUG [cloud.resource.ServerResourceBase] (TP-Processor18:null) Retrieving
network interface: xenbr1
2013-05-03 13:58:52,364 DEBUG [cloud.resource.ServerResourceBase] (TP-Processor18:null) Unable
to get network interface for xenbr1
2013-05-03 13:58:52,364 DEBUG [cloud.resource.ServerResourceBase] (TP-Processor18:null) Retrieving
network interface: null
2013-05-03 13:58:52,364 DEBUG [cloud.resource.ServerResourceBase] (TP-Processor18:null) Retrieving
network interface: null
2013-05-03 13:58:52,364 ERROR [cloud.resource.ServerResourceBase] (TP-Processor18:null) Nics
are not configured!
2013-05-03 13:58:52,366 INFO  [cloud.resource.ServerResourceBase] (TP-Processor18:null) Designating
private to be nic eth0
2013-05-03 13:58:52,409 DEBUG [storage.resource.NfsSecondaryStorageResource] (TP-Processor18:null)
Executing: /bin/bash -c ln -sf /mnt/SecStorage /var/www/html/copy 
2013-05-03 13:58:52,472 DEBUG [storage.resource.NfsSecondaryStorageResource] (TP-Processor18:null)
Execution is successful.
2013-05-03 13:58:52,472 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Startup
cmds : 1
2013-05-03 13:58:52,472 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Cmd:
StartupCommand{type=SecondaryStorage, dataCenter='null', pod='null', cluster='null', guid='null',
name='null', id=null,
 version='null', iqn='NoIqn', publicIpAddress='null', publicNetmask='null', publicMacAddress='null',
privateIpAddress='10.200.194.26', privateMacAddress='00:50:56:a9:50:b6', privateNetmask='255.255.254.0',
storage
IpAddress='10.200.194.26', storageNetmask='255.255.254.0', storageMacAddress='00:50:56:a9:50:b6',
storageIpAddressDeux='null', storageMacAddressDeux='null', storageNetmaskDeux='null', agentTag='null',
resourceName
='null', gatewayIpAddress='null'}
2013-05-03 13:58:52,473 DEBUG [agent.transport.Request] (TP-Processor18:null) Seq -1--1: Startup
request from directly connected host:  { Cmd , MgmtId: -1, via: -1, Ver: v1, Flags: 111, [{"type":"SecondaryStorage"
,"iqn":"NoIqn","privateIpAddress":"10.200.194.26","privateMacAddress":"00:50:56:a9:50:b6","privateNetmask":"255.255.254.0","storageIpAddress":"10.200.194.26","storageNetmask":"255.255.254.0","storageMacAddress":"0
0:50:56:a9:50:b6","wait":0}] }
2013-05-03 13:58:52,480 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Create
cmd com.cloud.agent.api.StartupSecondaryStorageCommand
2013-05-03 13:58:52,492 WARN  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Unable
to connect due to 
java.lang.IllegalArgumentException: Host 10.200.194.26 sent incorrect data center: null
        at com.cloud.resource.ResourceManagerImpl.createHostVO(ResourceManagerImpl.java:1767)
        at com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1947)
        at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:848)
        at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:628)
        at org.apache.cloudstack.api.command.admin.host.AddSecondaryStorageCmd.execute(AddSecondaryStorageCmd.java:80)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
        at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
        at com.cloud.api.ApiServer.handleRequest(ApiServer.java:359)
        at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:315)
        at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)

in the api.log file :

GET command=addSecondaryStorage&zoneId=dda44dc5-1084-442a-b946-00c66387a45a&url=nfs%3A%2F%2Frr.xxxxxx%2FifsxxxxxxFsecondarystorage&response=json&sessionkey=b2ORpgifHR1biF2EqFGYBiSHsN8%3D&_=1367582345384
530 Failed to add secondary storage

in the database :

select id from data_center where uuid = 'dda44dc5-1084-442a-b946-00c66387a45a' ;
+----+
| id |
+----+
|  1 |
+----+

It seems CS doesn't find the Zone using the uuid (from the api command).
                
      was (Author: nlamirault):
    I add some logs :
2013-05-03 13:58:51,752 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Trying
to add a new host at nfs://xxxxxxxxx/secondarystorage in data center 1
2013-05-03 13:58:52,409 DEBUG [storage.resource.NfsSecondaryStorageResource] (TP-Processor18:null)
Executing: /bin/bash -c ln -sf /mnt/SecStorage /var/www/html/copy 
2013-05-03 13:58:52,472 DEBUG [storage.resource.NfsSecondaryStorageResource] (TP-Processor18:null)
Execution is successful.
2013-05-03 13:58:52,472 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Startup
cmds : 1
2013-05-03 13:58:52,472 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Cmd:
StartupCommand{type=SecondaryStorage, dataCenter='null', pod='null', cluster='null', guid='null',
name='null', id=null,
 version='null', iqn='NoIqn', publicIpAddress='null', publicNetmask='null', publicMacAddress='null',
privateIpAddress='10.200.194.26', privateMacAddress='00:50:56:a9:50:b6', privateNetmask='255.255.254.0',
storage
IpAddress='10.200.194.26', storageNetmask='255.255.254.0', storageMacAddress='00:50:56:a9:50:b6',
storageIpAddressDeux='null', storageMacAddressDeux='null', storageNetmaskDeux='null', agentTag='null',
resourceName
='null', gatewayIpAddress='null'}
2013-05-03 13:58:52,473 DEBUG [agent.transport.Request] (TP-Processor18:null) Seq -1--1: Startup
request from directly connected host:  { Cmd , MgmtId: -1, via: -1, Ver: v1, Flags: 111, [{"type":"SecondaryStorage"
,"iqn":"NoIqn","privateIpAddress":"10.200.194.26","privateMacAddress":"00:50:56:a9:50:b6","privateNetmask":"255.255.254.0","storageIpAddress":"10.200.194.26","storageNetmask":"255.255.254.0","storageMacAddress":"0
0:50:56:a9:50:b6","wait":0}] }
2013-05-03 13:58:52,480 INFO  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Create
cmd com.cloud.agent.api.StartupSecondaryStorageCommand
2013-05-03 13:58:52,492 WARN  [cloud.resource.ResourceManagerImpl] (TP-Processor18:null) Unable
to connect due to 
java.lang.IllegalArgumentException: Host 10.200.194.26 sent incorrect data center: null
        at com.cloud.resource.ResourceManagerImpl.createHostVO(ResourceManagerImpl.java:1767)
        at com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1947)
        at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:848)
        at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:628)
        at org.apache.cloudstack.api.command.admin.host.AddSecondaryStorageCmd.execute(AddSecondaryStorageCmd.java:80)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
        at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
        at com.cloud.api.ApiServer.handleRequest(ApiServer.java:359)
        at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:315)
        at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)

in the api.log file :

GET command=addSecondaryStorage&zoneId=dda44dc5-1084-442a-b946-00c66387a45a&url=nfs%3A%2F%2Frr.xxxxxx%2FifsxxxxxxFsecondarystorage&response=json&sessionkey=b2ORpgifHR1biF2EqFGYBiSHsN8%3D&_=1367582345384
530 Failed to add secondary storage

in the database :

select id from data_center where uuid = 'dda44dc5-1084-442a-b946-00c66387a45a' ;
+----+
| id |
+----+
|  1 |
+----+

It seems CS doesn't find the Zone using the uuid (from the api command).
                  
> Failed to create secondary storage
> ----------------------------------
>
>                 Key: CLOUDSTACK-528
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-528
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Storage Controller
>    Affects Versions: 4.0.0
>         Environment: CentOS 
>            Reporter: Tamas Monos
>            Priority: Blocker
>             Fix For: 4.0.2
>
>
> Hi,
> I cannot add secondary storage to my existing Zone. This prevents the platform from operating.
> API call via GUI captured by tcpdump:
> http://sandbox.veber.co.uk/client/api?command=addSecondaryStorage&zoneId=a8d21e6d-bc9e-4dc2-b996-fc22727cd9c1&url=nfs%3A%2F%2F192.168.1.2%2Fwatford_cloud&response=json&sessionkey=DGl3OmkfQRGwuIDZFoOpoF2WBJU%3D&_=1353673985114
> Result:
> Unable to connect due to 
> java.lang.IllegalArgumentException: Host 192.168.1.4 sent incorrect data center: null
>         at com.cloud.resource.ResourceManagerImpl.createHostVO(ResourceManagerImpl.java:1501)
>         at com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1629)
>         at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:720)
>         at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:551)
>         at com.cloud.api.commands.AddSecondaryStorageCmd.execute(AddSecondaryStorageCmd.java:79)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
>         at com.cloud.api.ApiServer.queueCommand(ApiServer.java:543)
>         at com.cloud.api.ApiServer.handleRequest(ApiServer.java:422)
>         at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:304)
>         at com.cloud.api.ApiServlet.doGet(ApiServlet.java:63)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>         at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>         at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>         at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>         at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>         at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>         at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>         at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>         at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
>         at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
>         at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2012-11-23 12:32:58,518 INFO  [cloud.resource.ResourceManagerImpl] (catalina-exec-23:null)
server resources successfully discovered by SecondaryStorage
> 2012-11-23 12:32:58,518 WARN  [cloud.api.ApiDispatcher] (catalina-exec-23:null) class
com.cloud.api.ServerApiException : Failed to add secondary storage
> Debug:
> mysql> select uuid from data_center where name='Watford';
> +--------------------------------------+
> | uuid                                 |
> +--------------------------------------+
> | a8d21e6d-bc9e-4dc2-b996-fc22727cd9c1 |
> +--------------------------------------+
> The Zone ID matches but API throws exception while the IS is supplied in the query.
> Scenario:
> I have upgraded from 3.0.2 to 4.0.0. Due to the upgrade scripts did not update the systemVMs
to the new template I've applied a workaround and recreated the systemVMs with the new templates.
After that I've enabled storage clean-up and the cleanup script destroyed everything on my
primary storage including systemVMs. I wanted to re-create them but cannot add secondary storage
to the Zone.
> The applied workarounds during upgrade:
> 1. Add new vmware template to CS with name systemvm-vmware-4.0 (I have re-imported it)
2. Wait till the template is downloaded and installed successfully
> 3. Look up id of this template in DB (Name should match the input provided in step
> mysql> select id from `cloud`.`vm_template` where name = 
> mysql> 'systemvm-vmware-4.0' and removed is null;
> 4. Update template type to SYSTEM
> mysql> update `cloud`.`vm_template` set type='SYSTEM' where id = 
> mysql> <id-from-step3>;
> 5. Update template Id for all system Vms
> mysql> update `cloud`.`vm_instance` set vm_template_id = <id-from-step3> 
> mysql> where type <> 'User' and hypervisor_type = 'VMware';
> 6. Restart all system Vms

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message