cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Remi Bergsma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-8808) Successfully registered VHD template is downloaded again due to missing virtualsize property in template.properties
Date Fri, 04 Sep 2015 20:42:45 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Remi Bergsma updated CLOUDSTACK-8808:
-------------------------------------
    Description: 
We noticed all of our templates are downloaded again as soon as we restart SSVM, its Cloud
service or the management server it connects to.

A scan done by the SSVM (listvmtmplt.sh) returns the template, but it is rejected later (Post
download installation was not completed) because (Format is invalid) due to missing virtualSize
property in template.properties.

The initial registration did succeed however. I'd either want the registration to fail, or
it to succeed. Not first succeed (and spin VMs without a problem) then fail unexpectedly later.

This is the script processing the download:

services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java

 759     private List<String> listTemplates(String rootdir) {                      
  
 760         List<String> result = new ArrayList<String>();                  
        
 761                                                                                  
 762         Script script = new Script(listTmpltScr, s_logger);                      
 763         script.add("-r", rootdir);   

For example this becomes:
==> /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0

In this log file, it processes the output:
less /var/log/cloud/cloud.out

2015-09-04 08:39:54,622 WARN  [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null)
Post download installation was not completed for /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607

This error message is generated here:

services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
 
780         List<String> publicTmplts = listTemplates(templateDir);                
 
 781         for (String tmplt : publicTmplts) {                                      
 782             String path = tmplt.substring(0, tmplt.lastIndexOf(File.separator)); 
 783             TemplateLocation loc = new TemplateLocation(_storage, path);         
 784             try {                                                                
 785                 if (!loc.load()) {                                               
 786                     s_logger.warn("Post download installation was not completed for "
+ path);
 787                     // loc.purge();                                              
 788                     _storage.cleanup(path, templateDir);                         
 789                     continue;                                                    
 790                 }                                                                
 791             } catch (IOException e) {                                            
 792                 s_logger.warn("Unable to load template location " + path, e);    
 793                 continue;                                                        
 794             } 

In the logs this message is also seen:

MCCP-ADMIN-1|s-32436-VM CLOUDSTACK: 10:09:17,333  WARN TemplateLocation:196 - Format is invalid


It is generated here:

.//core/src/com/cloud/storage/template/TemplateLocation.java

192    public boolean addFormat(FormatInfo newInfo) {                               
193         deleteFormat(newInfo.format);                                            
194                                                                                  
195         if (!checkFormatValidity(newInfo)) {                                     
196             s_logger.warn("Format is invalid ");                                 
197             return false;                                                        
198         }                                                                        
199                                                                                  
200         _props.setProperty("virtualsize", Long.toString(newInfo.virtualSize));   
201         _formats.add(newInfo);                                                   
202         return true;                                                             
203     }                          

This returns false if checkFormatValidity is false.

checkFormatValidity:
209     protected boolean checkFormatValidity(FormatInfo info) {                     
210         return (info.format != null && info.size > 0 && info.virtualSize
> 0 && info.filename != null);
211     } 

This returns false if virtualSize is missing in template.properties. And indeed it is missing.

Examples:

Working KVM

root@s-44134-VM:/mnt/SecStorage/884db36b-cd00-3cf0-b812-831e9e50a4b3/template/tmpl/2/2932#
cat template.properties
#
#Thu Sep 03 11:41:44 UTC 2015
filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
id=2932
qcow2.size=1808203776
public=true
uniquename=2932-2-5c53bbe1-0c32-32a9-a82c-a5eefcb472e5
qcow2.virtualsize=21474836480
virtualsize=21474836480
checksum=4c28fc94a7a9cbe858918a97f3b78294
hvm=true
description=Centos7-x86_64-Sbp_cis-KVM release 2015-35 build
qcow2=true
qcow2.filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2

Working XenServer

cat /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/9/603/template.properties
filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
vhd=true
id=603
vhd.filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
public=false
uniquename=16447323-ae24-4956-ac67-316eed8a2626
vhd.virtualsize=107374182400
virtualsize=107374182400
checksum=
hvm=true
description=570a3650f0-c982-334a-81b1-e8cf961363e4
vhd.size=69028311552
size=69028311552


Not working XenServer

root@s-32435-VM:/mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607#
cat template.properties 
#
#Fri Sep 04 09:05:24 UTC 2015
filename=39794796-3f92-3168-9c4e-a64da5ee06e8.vhd
uniquename=1607-2-0a13984d-6724-3b9f-8bd1-2798ef859167
size=21475270656
checksum=6eb85d82cb18032eba0cc8fe8a84a583
description=Centos7-x86_64-Sbp_cis-XenServer release 2015-35 build
hvm=true
public=true
id=1607


Confirmed the virtualsize is missing. But why is this successfully registered in the first
place?


Steps to reproduce:
- register template
- have it install successfully
- check templates.properties file
  sometimes virtualsize is missing
- if it's there, remove it
- restart cloud service on SSVM
- the download will start again
  in my case virtualsize is still missing
- this becomes an endless loop.


  was:
We noticed all of our templates are downloaded again as soon as we restart SSVM, its Cloud
service or the management server it connects to.

A scan done by the SSVM (listvmtmplt.sh) returns the template, but it is rejected later (Post
download installation was not completed) because (Format is invalid) due to missing virtualSize
property in template.properties.

The initial registeration did succeed however. I'd either want the registration to fail, or
it to succeed. Not first succeed (and spin VMs without a problem) then fail unexpectedly later.

Logs:


services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java

 759     private List<String> listTemplates(String rootdir) {                      
  
 760         List<String> result = new ArrayList<String>();                  
        
 761                                                                                  
 762         Script script = new Script(listTmpltScr, s_logger);                      
 763         script.add("-r", rootdir);   

==> /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0

less /var/log/cloud/cloud.out

2015-09-04 08:39:54,622 WARN  [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null)
Post download installation was not completed for /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607

services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
 
780         List<String> publicTmplts = listTemplates(templateDir);                
 
 781         for (String tmplt : publicTmplts) {                                      
 782             String path = tmplt.substring(0, tmplt.lastIndexOf(File.separator)); 
 783             TemplateLocation loc = new TemplateLocation(_storage, path);         
 784             try {                                                                
 785                 if (!loc.load()) {                                               
 786                     s_logger.warn("Post download installation was not completed for "
+ path);
 787                     // loc.purge();                                              
 788                     _storage.cleanup(path, templateDir);                         
 789                     continue;                                                    
 790                 }                                                                
 791             } catch (IOException e) {                                            
 792                 s_logger.warn("Unable to load template location " + path, e);    
 793                 continue;                                                        
 794             } 

Log:
MCCP-ADMIN-1|s-32436-VM CLOUDSTACK: 10:09:17,333  WARN TemplateLocation:196 - Format is invalid



.//core/src/com/cloud/storage/template/TemplateLocation.java

192    public boolean addFormat(FormatInfo newInfo) {                               
193         deleteFormat(newInfo.format);                                            
194                                                                                  
195         if (!checkFormatValidity(newInfo)) {                                     
196             s_logger.warn("Format is invalid ");                                 
197             return false;                                                        
198         }                                                                        
199                                                                                  
200         _props.setProperty("virtualsize", Long.toString(newInfo.virtualSize));   
201         _formats.add(newInfo);                                                   
202         return true;                                                             
203     }                          

This returns false if checkFormatValidity is false.

checkFormatValidity:
209     protected boolean checkFormatValidity(FormatInfo info) {                     
210         return (info.format != null && info.size > 0 && info.virtualSize
> 0 && info.filename != null);
211     } 

This returns false if virtualSize is missing in template.properties. And indeed it is missing.

Examples:

Working KVM

root@s-44134-VM:/mnt/SecStorage/884db36b-cd00-3cf0-b812-831e9e50a4b3/template/tmpl/2/2932#
cat template.properties
#
#Thu Sep 03 11:41:44 UTC 2015
filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
id=2932
qcow2.size=1808203776
public=true
uniquename=2932-2-5c53bbe1-0c32-32a9-a82c-a5eefcb472e5
qcow2.virtualsize=21474836480
virtualsize=21474836480
checksum=4c28fc94a7a9cbe858918a97f3b78294
hvm=true
description=Centos7-x86_64-Sbp_cis-KVM release 2015-35 build
qcow2=true
qcow2.filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2

Working XenServer

cat /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/9/603/template.properties
filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
vhd=true
id=603
vhd.filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
public=false
uniquename=16447323-ae24-4956-ac67-316eed8a2626
vhd.virtualsize=107374182400
virtualsize=107374182400
checksum=
hvm=true
description=570a3650f0-c982-334a-81b1-e8cf961363e4
vhd.size=69028311552
size=69028311552


Not working XenServer

root@s-32435-VM:/mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607#
cat template.properties 
#
#Fri Sep 04 09:05:24 UTC 2015
filename=39794796-3f92-3168-9c4e-a64da5ee06e8.vhd
uniquename=1607-2-0a13984d-6724-3b9f-8bd1-2798ef859167
size=21475270656
checksum=6eb85d82cb18032eba0cc8fe8a84a583
description=Centos7-x86_64-Sbp_cis-XenServer release 2015-35 build
hvm=true
public=true
id=1607


Confirmed the virtualsize is missing. But why is this successfully registered in the first
place?


Steps to reproduce:
- register template
- have it install successfully
- check templates.properties file
  sometimes virtualsize is missing
- if it's there, remove it
- restart cloud service on SSVM
- the download will start again
  in my case virtualsize is still missing
- this becomes an endless loop.



> Successfully registered VHD template is downloaded again due to missing virtualsize property
in template.properties
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-8808
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8808
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Secondary Storage
>    Affects Versions: 4.4.4, 4.6.0
>         Environment: Seen on NFS as sec storage
>            Reporter: Remi Bergsma
>            Priority: Critical
>
> We noticed all of our templates are downloaded again as soon as we restart SSVM, its
Cloud service or the management server it connects to.
> A scan done by the SSVM (listvmtmplt.sh) returns the template, but it is rejected later
(Post download installation was not completed) because (Format is invalid) due to missing
virtualSize property in template.properties.
> The initial registration did succeed however. I'd either want the registration to fail,
or it to succeed. Not first succeed (and spin VMs without a problem) then fail unexpectedly
later.
> This is the script processing the download:
> services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
>  759     private List<String> listTemplates(String rootdir) {                 
       
>  760         List<String> result = new ArrayList<String>();             
             
>  761                                                                                
 
>  762         Script script = new Script(listTmpltScr, s_logger);                    
 
>  763         script.add("-r", rootdir);   
> For example this becomes:
> ==> /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0
> In this log file, it processes the output:
> less /var/log/cloud/cloud.out
> 2015-09-04 08:39:54,622 WARN  [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null)
Post download installation was not completed for /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607
> This error message is generated here:
> services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
>  
> 780         List<String> publicTmplts = listTemplates(templateDir);           
      
>  781         for (String tmplt : publicTmplts) {                                    
 
>  782             String path = tmplt.substring(0, tmplt.lastIndexOf(File.separator));

>  783             TemplateLocation loc = new TemplateLocation(_storage, path);       
 
>  784             try {                                                              
 
>  785                 if (!loc.load()) {                                             
 
>  786                     s_logger.warn("Post download installation was not completed
for " + path);
>  787                     // loc.purge();                                            
 
>  788                     _storage.cleanup(path, templateDir);                       
 
>  789                     continue;                                                  
 
>  790                 }                                                              
 
>  791             } catch (IOException e) {                                          
 
>  792                 s_logger.warn("Unable to load template location " + path, e);  
 
>  793                 continue;                                                      
 
>  794             } 
> In the logs this message is also seen:
> MCCP-ADMIN-1|s-32436-VM CLOUDSTACK: 10:09:17,333  WARN TemplateLocation:196 - Format
is invalid 
> It is generated here:
> .//core/src/com/cloud/storage/template/TemplateLocation.java
> 192    public boolean addFormat(FormatInfo newInfo) {                               
> 193         deleteFormat(newInfo.format);                                           

> 194                                                                                 

> 195         if (!checkFormatValidity(newInfo)) {                                    

> 196             s_logger.warn("Format is invalid ");                                

> 197             return false;                                                       

> 198         }                                                                       

> 199                                                                                 

> 200         _props.setProperty("virtualsize", Long.toString(newInfo.virtualSize));  

> 201         _formats.add(newInfo);                                                  

> 202         return true;                                                            

> 203     }                          
> This returns false if checkFormatValidity is false.
> checkFormatValidity:
> 209     protected boolean checkFormatValidity(FormatInfo info) {                    

> 210         return (info.format != null && info.size > 0 && info.virtualSize
> 0 && info.filename != null);
> 211     } 
> This returns false if virtualSize is missing in template.properties. And indeed it is
missing.
> Examples:
> Working KVM
> root@s-44134-VM:/mnt/SecStorage/884db36b-cd00-3cf0-b812-831e9e50a4b3/template/tmpl/2/2932#
cat template.properties
> #
> #Thu Sep 03 11:41:44 UTC 2015
> filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
> id=2932
> qcow2.size=1808203776
> public=true
> uniquename=2932-2-5c53bbe1-0c32-32a9-a82c-a5eefcb472e5
> qcow2.virtualsize=21474836480
> virtualsize=21474836480
> checksum=4c28fc94a7a9cbe858918a97f3b78294
> hvm=true
> description=Centos7-x86_64-Sbp_cis-KVM release 2015-35 build
> qcow2=true
> qcow2.filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
> Working XenServer
> cat /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/9/603/template.properties
> filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
> vhd=true
> id=603
> vhd.filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
> public=false
> uniquename=16447323-ae24-4956-ac67-316eed8a2626
> vhd.virtualsize=107374182400
> virtualsize=107374182400
> checksum=
> hvm=true
> description=570a3650f0-c982-334a-81b1-e8cf961363e4
> vhd.size=69028311552
> size=69028311552
> Not working XenServer
> root@s-32435-VM:/mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607#
cat template.properties 
> #
> #Fri Sep 04 09:05:24 UTC 2015
> filename=39794796-3f92-3168-9c4e-a64da5ee06e8.vhd
> uniquename=1607-2-0a13984d-6724-3b9f-8bd1-2798ef859167
> size=21475270656
> checksum=6eb85d82cb18032eba0cc8fe8a84a583
> description=Centos7-x86_64-Sbp_cis-XenServer release 2015-35 build
> hvm=true
> public=true
> id=1607
> Confirmed the virtualsize is missing. But why is this successfully registered in the
first place?
> Steps to reproduce:
> - register template
> - have it install successfully
> - check templates.properties file
>   sometimes virtualsize is missing
> - if it's there, remove it
> - restart cloud service on SSVM
> - the download will start again
>   in my case virtualsize is still missing
> - this becomes an endless loop.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message