cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Min Chen (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (CLOUDSTACK-4250) [Object_store_refactor] Register ISO fails in multizone environment if downloading to one of the zone fails
Date Mon, 12 Aug 2013 17:04:49 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Min Chen reassigned CLOUDSTACK-4250:
------------------------------------

    Assignee: Min Chen
    
> [Object_store_refactor] Register ISO fails in multizone environment if downloading to
one of the zone fails
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-4250
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4250
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: ISO, Storage Controller, Template
>    Affects Versions: 4.2.0
>         Environment: Latest build from ACS 4.2 branch.
> Storage: NFS for both primary and secondary storage
>            Reporter: Sanjeev N
>            Assignee: Min Chen
>            Priority: Critical
>             Fix For: 4.2.0
>
>         Attachments: cloud.dmp, management-server.rar
>
>
> Registering ISO/template with AllZones yes fails If the setup has more than one zone
and one of the zones is disabled and SSVM is not running.
> Steps to Reproduce:
> ================
> 1.Bring up CS with more than one zone say zone1 and zone2 both using NFS for primary
and secondary storage
> 2.Disable zone2 and stop SSVM in it
> 3.Register ISO/Template in zone1 and choose zone= All Zones in UI while registering iso
> Result:
> ======
> Template download was succeeded in zone1 but it was timed out in zone2. Timeout in downloading
template in zone2 deleted the template from both the zones
> Observations:
> ===========
> Log snippet from management server log file:
> 2013-08-12 20:58:37,978 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null) ===START===
 10.146.0.20 -- GET  command=registerIso&response=json&sessionkey=G1sYAr9SNj4GF7irv5eSdcqDB10%3D&name=all&displayText=all&url=http%3A%2F%2F10.147.28.7%2Ftemplates%2Fvmware%2Fdummy.iso&zoneid=-1&isextractable=true&bootable=true&osTypeId=12&ispublic=true&isfeatured=true&_=1376301850336
> 2013-08-12 20:58:38,171 DEBUG [storage.image.TemplateDataFactoryImpl] (catalina-exec-22:null)
template 220 is not in store:6, type:Image
> 2013-08-12 20:58:38,250 DEBUG [storage.image.TemplateDataFactoryImpl] (catalina-exec-22:null)
template 220 is already in store:6, type:Image
> 2013-08-12 20:58:38,286 DEBUG [storage.image.BaseImageStoreDriverImpl] (catalina-exec-22:null)
Downloading template to data store 6
> 2013-08-12 20:58:38,313 DEBUG [storage.image.BaseImageStoreDriverImpl] (catalina-exec-22:null)
Performing image store createTemplate async callback
> 2013-08-12 20:58:38,340 DEBUG [cloudstack.storage.RemoteHostEndPoint] (catalina-exec-22:null)
Sending command org.apache.cloudstack.storage.command.DownloadCommand to host: 7
> 2013-08-12 20:58:38,344 DEBUG [agent.transport.Request] (catalina-exec-22:null) Seq 7-1199968632:
Sending  { Cmd , MgmtId: 6851580133501, via: 7, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DownloadCommand":{"hvm":true,"description":"all","maxDownloadSizeInBytes":53687091200,"id":220,"resourceType":"TEMPLATE","installPath":"template/tmpl/9/220","_store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/manasa/secondaryVMw","_role":"Image"}},"url":"http://10.147.28.7/templates/vmware/dummy.iso","format":"ISO","accountId":9,"name":"220-9-41f76fdf-af7e-3d03-91aa-f7fd95fcdf02","secUrl":"nfs://10.147.28.7/export/home/manasa/secondaryVMw","wait":0}}]
}
> 2013-08-12 20:58:38,350 DEBUG [storage.image.TemplateDataFactoryImpl] (catalina-exec-22:null)
template 220 is not in store:9, type:Image
> 2013-08-12 20:58:38,390 DEBUG [storage.image.TemplateDataFactoryImpl] (catalina-exec-22:null)
template 220 is already in store:9, type:Image
> 2013-08-12 20:58:38,431 DEBUG [storage.image.BaseImageStoreDriverImpl] (catalina-exec-22:null)
Downloading template to data store 9
> 2013-08-12 20:58:38,447 INFO  [storage.endpoint.DefaultEndPointSelector] (catalina-exec-22:null)
No running ssvm is found, so command will be sent to LocalHostEndPoint
> 2013-08-12 20:58:38,450 DEBUG [storage.image.BaseImageStoreDriverImpl] (catalina-exec-22:null)
Performing image store createTemplate async callback
> 2013-08-12 20:58:38,471 DEBUG [storage.image.TemplateDataFactoryImpl] (catalina-exec-22:null)
template 220 is not in store:2, type:Image
> 2013-08-12 20:58:38,499 DEBUG [storage.image.TemplateDataFactoryImpl] (catalina-exec-22:null)
template 220 is already in store:2, type:Image
> 2013-08-12 20:58:38,524 DEBUG [storage.image.BaseImageStoreDriverImpl] (catalina-exec-22:null)
Downloading template to data store 2
> In the above log, store 9 was present in a zone which was in disabled state and SSVM
was in stopped state. But still CS initiated download via  LocalHostEndPoint
> Q: What is LocalHostEndPoint? Is it management server? If yes why do we initiate download
via management server?
> 2013-08-12 20:58:38,540 DEBUG [storage.image.BaseImageStoreDriverImpl] (catalina-exec-22:null)
Performing image store createTemplate async callback
> 2013-08-12 20:58:38,556 DEBUG [cloudstack.storage.RemoteHostEndPoint] (catalina-exec-22:null)
Sending command org.apache.cloudstack.storage.command.DownloadCommand to host: 14
> 2013-08-12 20:58:38,559 DEBUG [agent.transport.Request] (catalina-exec-22:null) Seq 14-670765384:
Sending  { Cmd , MgmtId: 6851580133501, via: 14, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DownloadCommand":{"hvm":true,"description":"all","maxDownloadSizeInBytes":53687091200,"id":220,"resourceType":"TEMPLATE","installPath":"template/tmpl/9/220","_store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/manasa/secondaryxen","_role":"Image"}},"url":"http://10.147.28.7/templates/vmware/dummy.iso","format":"ISO","accountId":9,"name":"220-9-41f76fdf-af7e-3d03-91aa-f7fd95fcdf02","secUrl":"nfs://10.147.28.7/export/home/manasa/secondaryxen","wait":0}}]
}
> 2013-08-12 20:58:38,594 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null) ===END===
 10.146.0.20 -- GET  command=registerIso&response=json&sessionkey=G1sYAr9SNj4GF7irv5eSdcqDB10%3D&name=all&displayText=all&url=http%3A%2F%2F10.147.28.7%2Ftemplates%2Fvmware%2Fdummy.iso&zoneid=-1&isextractable=true&bootable=true&osTypeId=12&ispublic=true&isfeatured=true&_=1376301850336
> 2013-08-12 20:58:42,059 DEBUG [agent.transport.Request] (AgentManager-Handler-12:null)
Seq 7-1199968632: Processing:  { Ans: , MgmtId: 6851580133501, via: 7, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"edf48a3a-24a2-4ea5-8863-87cba1a31971","downloadPct":100,"errorString":"Download
success, starting install ","downloadStatus":"DOWNLOAD_IN_PROGRESS","downloadPath":"/mnt/SecStorage/0e8da06e-0788-3efb-86a6-b0705a2205d3/template/tmpl/9/220/dnld6918760524895984102tmp_","installPath":"template/tmpl/9/220/220-9-41f76fdf-af7e-3d03-91aa-f7fd95fcdf02.iso","templateSize":0,"templatePhySicalSize":0,"checkSum":"18425b903c6969ed63f328ef4fd346af","result":true,"details":"Download
success, starting install ","wait":0}}] }
> 2013-08-12 20:58:42,061 DEBUG [agent.transport.Request] (AgentManager-Handler-15:null)
Seq 14-670765384: Processing:  { Ans: , MgmtId: 6851580133501, via: 14, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"4dafa883-a1bf-4693-a463-049120455173","downloadPct":100,"errorString":"Download
success, starting install ","downloadStatus":"DOWNLOAD_IN_PROGRESS","downloadPath":"/mnt/SecStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/9/220/dnld2814394365859359939tmp_","installPath":"template/tmpl/9/220/220-9-41f76fdf-af7e-3d03-91aa-f7fd95fcdf02.iso","templateSize":0,"templatePhySicalSize":0,"checkSum":"18425b903c6969ed63f328ef4fd346af","result":true,"details":"Download
success, starting install ","wait":0}}] }
> 2013-08-12 20:58:42,623 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:null)
SeqA 4-36208: Processing Seq 4-36208:  { Cmd , MgmtId: -1, via: 4, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
 \"connections\": []\n}","wait":0}}] }
> 2013-08-12 20:58:42,632 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:null)
SeqA 4-36208: Sending Seq 4-36208:  { Ans: , MgmtId: 6851580133501, via: 4, Ver: v1, Flags:
100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2013-08-12 20:58:43,257 DEBUG [cloud.server.StatsCollector] (StatsCollector-1:null) StorageCollector
is running...
> 2013-08-12 20:58:43,817 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 14-670765385:
Received:  { Ans: , MgmtId: 6851580133501, via: 14, Ver: v1, Flags: 10, { GetStorageStatsAnswer
} }
> 2013-08-12 20:58:43,934 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 7-1199968633:
Received:  { Ans: , MgmtId: 6851580133501, via: 7, Ver: v1, Flags: 10, { GetStorageStatsAnswer
} }
> 2013-08-12 20:58:43,938 INFO  [storage.endpoint.DefaultEndPointSelector] (StatsCollector-1:null)
No running ssvm is found, so command will be sent to LocalHostEndPoint
> Before deleting ISO from CS I could see the following timeout messages:
> 2013-08-12 20:59:38,450 DEBUG [storage.download.DownloadListener] (Timer-25:null) timeout:
transitioning to download error state, currstate=NOT_DOWNLOADED, TEMPLATE: 220 at host 0
> 2013-08-12 20:59:38,450 DEBUG [storage.image.BaseImageStoreDriverImpl] (Timer-25:null)
Performing image store createTemplate async callback
> 2013-08-12 20:59:38,543 WARN  [storage.download.DownloadListener] (Timer-25:null) Entering
download error state: timeout waiting for response from storage host, TEMPLATE: 220 at host
0
> Attaching management server log file and cloud DB.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message