cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhinav Roy (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (CLOUDSTACK-209) Upgrade from CS-3.0.2 to ASF 4.0 fails with com.cloud.utils.exception.CloudRuntimeException: Unable to find key inject script scripts/vm/systemvm/injectkeys.sh
Date Wed, 03 Oct 2012 11:24:07 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Abhinav Roy reopened CLOUDSTACK-209:
------------------------------------


Re-opening the issue due to following observations made during upgrade done using the latest
build
http://jenkins.cloudstack.org/job/build-4.0-rhel63/lastSuccessfulBuild/artifact/CloudStack-oss-4.0.0-356.tar.bz2
===========================================
1. After the upgrade we have
    -----------------------------------------
    Installed:
    cloud-scripts.x86_64 0:4.0.0-0.356.el6.4.0
    Replaced/Removed:
    cloud-agent-scripts.x86_64 0:3.0.2-1.el6

Now, due to this we don't have all the scripts in the /usr/lib64/cloud/common/scripts folder
:

[root@burnank CloudStack-oss-4.0.0-356]# ls /usr/lib64/cloud/common/scripts/vm/
hypervisor


So, to get all the scripts we try to install cloud-scripts package again, but since it is
already installed, it can't  be installed again :

[root@burnank CloudStack-oss-4.0.0-356]# yum install cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.iitm.ac.in
 * extras: mirrors.sin3.sg.voxel.net
 * updates: ftp.iitm.ac.in
Setting up Install Process
Examining cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm: cloud-scripts-4.0.0-0.356.el6.4.0.x86_64
cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm: does not update installed package.
Error: Nothing to do.

So, we go to step 2.


2. Uninstall cloud-scripts and then install again
    ------------------------------------------------

    Now, uninstalling cloud-scripts uninstalls 4 other packages as dependencies :

Removed:
  cloud-scripts.x86_64 0:4.0.0-0.356.el6.4.0                                             
                                                                                         
         

Dependency Removed:
  cloud-client.x86_64 0:4.0.0-0.356.el6.4.0     cloud-client-ui.x86_64 0:4.0.0-0.356.el6.4.0
    cloud-server.x86_64 0:4.0.0-0.356.el6.4.0     cloud-setup.x86_64 0:4.0.0-0.356.el6.4.0
   

     
    Install cloud-scripts and 4 other packages which got removed 
    
Now , we can see all the scripts present

[root@burnank CloudStack-oss-4.0.0-356]# ls /usr/lib64/cloud/common/scripts/vm/systemvm/
id_rsa.cloud  injectkeys.sh

3. We do the same on our KVM host. 
    ------------------------------------------------
    Here, while uninstalling cloud-scripts , cloud-agent also gets removed so, we install
both of them again.

4. Now, the upgrade process is over, so we start the cloud-agent and cloud-management services.

5. The upgrade goes fine, there are no errors.

Post Upgrade issues :
=================================

1. The host is in disconnected state, it is not being recognied , reconnecting gives the following
exception :

2012-10-03 15:41:27,118 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-4:job-13) Executing
com.cloud.api.commands.ReconnectHostCmd for job-13
2012-10-03 15:41:27,125 INFO  [agent.manager.AgentManagerImpl] (Job-Executor-4:job-13) Unable
to disconnect host because it is not in the correct state: host=1; Status=Disconnected
2012-10-03 15:41:27,126 WARN  [api.commands.ReconnectHostCmd] (Job-Executor-4:job-13) Exception:
com.cloud.api.ServerApiException
        at com.cloud.api.commands.ReconnectHostCmd.execute(ReconnectHostCmd.java:108)    
   at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
       at java.lang.Thread.run(Thread.java:679)
2012-10-03 15:41:27,127 WARN  [cloud.api.ApiDispatcher] (Job-Executor-4:job-13) class com.cloud.api.ServerApiException
: null
2012-10-03 15:41:27,127 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-4:job-13) Complete
async job-13, jobStatus: 2, resultCode: 530, result: Error Code: 534 Error text: null
2012-10-03 15:41:32,180 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-24:null) Async
job-13 completed


2. The VMs can not be instantiated as the host is not in UP state. To reconnect the host I
tried 
     
    [root@rajesh-kvm3 ~]# cloud-setup-agent
Welcome to the CloudStack Agent Setup:
Please input the Management Server Hostname/IP-Address:[localhost]10.102.125.218
Please input the Zone Id:[default]
Please input the Pod Id:[default]
Please input the Cluster Id:[default]
Please choose which network used to create VM:[cloudbr0]
Starting to configure your system:
Configure Cgroup ...          [OK]
Configure SElinux ...         [OK]
Configure Network ...         [OK]
Configure Libvirt ...         [OK]
Configure Firewall ...        [OK]
Configure Nfs ...             [OK]
Configure cloudAgent ...      [OK]
CloudStack Agent setup is done!


But this didn't reconnect the host, it added the same host as the new one while the same host
was also present in disconnected state.

But again the VMs were not getting deployed as the host was not recognising the secondary
storage vm etc.

---------------------------------------------------------------------------------------------------------

So because of the above mentioned issues we need to first of all get our packaging right,
the install/uninstall of the packages should happen in such a way that the user doesn't need
to install/remove any package manually. Just using the upgrade utility in the install.sh script
should be enough, both in the case of management server and the KVM host.
 
                
> Upgrade from CS-3.0.2 to ASF 4.0 fails with com.cloud.utils.exception.CloudRuntimeException:
Unable to find key inject script scripts/vm/systemvm/injectkeys.sh
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-209
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-209
>             Project: CloudStack
>          Issue Type: Bug
>          Components: Install and Setup, Management Server
>    Affects Versions: pre-4.0.0
>         Environment: MS : Rhel 6.2
> HOST : KVM ( Rhel 6.2)
> BUILDS : 
> CS-3.0.2 - CloudStack-3.0.2-1-rhel6.2.tar.gz
> ASF 4.0 -  CloudStack-oss-4.0.0-187.tar.bz2
>            Reporter: Abhinav Roy
>            Assignee: Abhinav Roy
>            Priority: Blocker
>             Fix For: pre-4.0.0
>
>         Attachments: agent.log, api-server.log, management-server.log, SQL_09_26.dmp
>
>
> Steps :
> ==============================
> 1. Deploy an advanced zone setup with CS-3.0.2 and KVM host (Rhel 6.2)
> 2. Create a VM instance.
> 3. Stop Management server.
> 4. Upgrade to ASF 4.0
> 5. Stop agent services on the host.
> 6. Upgrade to ASF 4.0
> 7. Start agent on the host.
> 8. Start management server services.
> Expected Behaviour :
> ==============================
> The upgrade should happen smoothly without any error.
> Observed Behaviour :
> ==============================
> 1. After executing above steps, following exception is seen in the logs
>     2012-09-26 17:44:41,571 DEBUG [utils.script.Script] (main:null) Looking for scripts/vm/systemvm/injectkeys.sh
in /var/lib/cloud/management/./scripts/vm/systemvm/injectkeys.sh
> 2012-09-26 17:44:41,571 DEBUG [utils.script.Script] (main:null) Looking for scripts/vm/systemvm/injectkeys.sh
in /var/lib/cloud/management/./scripts/vm/systemvm/injectkeys.sh
> 2012-09-26 17:44:41,571 WARN  [utils.script.Script] (main:null) Unable to find script
scripts/vm/systemvm/injectkeys.sh
> 2012-09-26 17:44:41,574 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in the classpath
> 2012-09-26 17:44:41,574 DEBUG [utils.script.Script] (main:null) System resource: null
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Current binaries reside
at /usr/share/java
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/share/java/vms/systemvm.iso
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/share/java/vms/systemvm.iso
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/share/vms/systemvm.iso
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/vms/systemvm.iso
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /vms/systemvm.iso
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null) Searching in environment.properties
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null) environment.properties
says scripts should be in /usr/lib64/cloud/common
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/lib64/cloud/common/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/lib64/cloud/common/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/lib64/cloud/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/lib64/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /usr/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Searching in the current
directory
> 2012-09-26 17:44:41,579 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /var/lib/cloud/management/./vms/systemvm.iso
> 2012-09-26 17:44:41,579 DEBUG [utils.script.Script] (main:null) Looking for vms/systemvm.iso
in /var/lib/cloud/management/./vms/systemvm.iso
> 2012-09-26 17:44:41,579 WARN  [utils.script.Script] (main:null) Unable to find script
vms/systemvm.iso
> 2012-09-26 17:44:41,582 ERROR [cloud.servlet.CloudStartupServlet] (main:null) Exception
starting management server 
> com.cloud.utils.exception.CloudRuntimeException: Unable to find key inject script scripts/vm/systemvm/injectkeys.sh
>         at com.cloud.server.ConfigurationServerImpl.updateKeyPairs(ConfigurationServerImpl.java:675)
>         at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
>         at com.cloud.server.ConfigurationServerImpl.persistDefaultValues(ConfigurationServerImpl.java:265)
>         at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
>         at com.cloud.servlet.CloudStartupServlet.init(CloudStartupServlet.java:47)
>         at javax.servlet.GenericServlet.init(GenericServlet.java:212)
>         at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
>         at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993)
>         at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4187)
>         at org.apache.catalina.core.StandardContext.start(StandardContext.java:4496)
>         at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>         at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>         at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>         at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>         at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>         at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>         at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>         at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>         at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
>         at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>         at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>         at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>         at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>         at org.apache.catalina.core.StandardService.start(StandardService.java:516)
>         at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>         at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> 2. This exception I feel is because of the change in script paths recently and is independent
of the host hypervisor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message