Return-Path: X-Original-To: apmail-incubator-cloudstack-issues-archive@minotaur.apache.org Delivered-To: apmail-incubator-cloudstack-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 102B39EBC for ; Mon, 4 Mar 2013 08:41:14 +0000 (UTC) Received: (qmail 25233 invoked by uid 500); 4 Mar 2013 08:41:13 -0000 Delivered-To: apmail-incubator-cloudstack-issues-archive@incubator.apache.org Received: (qmail 25175 invoked by uid 500); 4 Mar 2013 08:41:13 -0000 Mailing-List: contact cloudstack-issues-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cloudstack-dev@incubator.apache.org Delivered-To: mailing list cloudstack-issues@incubator.apache.org Received: (qmail 25161 invoked by uid 99); 4 Mar 2013 08:41:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Mar 2013 08:41:13 +0000 Date: Mon, 4 Mar 2013 08:41:12 +0000 (UTC) From: "Prasanna Santhanam (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CLOUDSTACK-1430) KVM: systemVMs failed to come up on 4.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592067#comment-13592067 ] Prasanna Santhanam commented on CLOUDSTACK-1430: ------------------------------------------------ I was able to write into a file named 'test' in the mount: [root@apache-83-1 21ce6784-c950-3f15-a29b-8da8bd65bfe7]# ls -la total 708956 drwxr-xr-x. 3 nobody nobody 4096 Mar 4 01:42 . drwxr-xr-x. 4 root root 4096 Mar 4 08:25 .. -rwxr--r--. 1 nobody nobody 725811200 Mar 4 01:35 45936281-102d-435d-9be4-b1acd66a4a73 -rwxr--r--. 1 nobody nobody 262144 Mar 4 01:35 faa5bbcb-109b-4b2f-88cd-621d1aba7603 drwxr-xr-x. 2 nobody nobody 4096 Mar 4 01:35 KVMHA -rw-r--r--. 1 nobody nobody 6 Mar 4 01:42 test [root@apache-83-1 21ce6784-c950-3f15-a29b-8da8bd65bfe7]# cat test test There is no patchdisk for s-X-VM or v-X-VM at the location. The content of the libvirt.log is as follows: 2013-03-04 08:24:29.735+0000: 6456: info : libvirt version: 0.9.10, package: 21.el6_3.8 (CentOS BuildSystem , 2013-01-28-19:24:16, c6b10.bsys.dev.centos.org) 2013-03-04 08:24:29.735+0000: 6456: error : virCommandWait:2304 : internal error Child process (/sbin/iptables --table filter --delete INPUT --in-interface virbr0 --protocol udp --destination-port 53 --jump ACCEPT) status unexpected: exit status 4 2013-03-04 08:24:29.761+0000: 6456: error : virCommandWait:2304 : internal error Child process (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol udp --destination-port 53 --jump ACCEPT) status unexpected: exit status 4 2013-03-04 08:24:29.761+0000: 6456: error : networkAddGeneralIptablesRules:1318 : failed to add iptables rule to allow DNS requests from 'virbr0' 2013-03-04 08:24:29.859+0000: 6456: warning : virSecurityManagerNew:129 : Configured security driver "none" disables default policy to create confined guests 2013-03-04 08:26:11.681+0000: 6447: error : virFileOpenForceOwnerMode:796 : cannot chown '/mnt/21ce6784-c950-3f15-a29b-8da8bd65bfe7/v-2-VM-patchdisk' to (0, 0): Invalid argument 2013-03-04 08:26:11.681+0000: 6447: error : virStorageBackendCreateRaw:401 : cannot create path '/mnt/21ce6784-c950-3f15-a29b-8da8bd65bfe7/v-2-VM-patchdisk': Invalid argument 2013-03-04 08:26:12.111+0000: 6450: error : virFileOpenForceOwnerMode:796 : cannot chown '/mnt/21ce6784-c950-3f15-a29b-8da8bd65bfe7/s-1-VM-patchdisk' to (0, 0): Invalid argument 2013-03-04 08:26:12.111+0000: 6450: error : virStorageBackendCreateRaw:401 : cannot create path '/mnt/21ce6784-c950-3f15-a29b-8da8bd65bfe7/s-1-VM-patchdisk': Invalid argument 2013-03-04 08:26:26.096+0000: 6447: error : virFileOpenForceOwnerMode:796 : cannot chown '/mnt/21ce6784-c950-3f15-a29b-8da8bd65bfe7/s-1-VM-patchdisk' to (0, 0): Invalid argument > KVM: systemVMs failed to come up on 4.1 > --------------------------------------- > > Key: CLOUDSTACK-1430 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1430 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the default.) > Components: KVM, Management Server > Affects Versions: 4.1.0 > Reporter: Prasanna Santhanam > Assignee: Marcus Sorensen > Priority: Blocker > Attachments: debugagentlogs.tar.bz2, mslogs.tar.bz2 > > > This is from the latest packages from jenkins.cs.o. > http://jenkins.cloudstack.org/job/package-rhel63-4.1/101/ > When the management server tries to bring up system vms, following exception is noted. > Agent is up and running and cloudbr0 is the device for public, private traffic > [root@apache-81-3 ~]# cat /etc/cloudstack/agent/agent.properties | grep cloudbr0 > guest.network.device=cloudbr0 > private.network.device=cloudbr0 > public.network.device=cloudbr0 > 2013-02-27 15:28:49,828 WARN [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Exception while trying to start secondary storage vm > com.cloud.exception.AgentUnavailableException: Resource [Host:3] is unreachable: Host 3: Unable to start instance due to Unable to get answer that is of class com.cloud.agent.api.StartAnswer > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:831) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:461) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:454) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319) > at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80) > at com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:37) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) > at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) > at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) > at sun.proxy.$Proxy193.start(Unknown Source) > at com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:264) > at com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:689) > at com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1300) > at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:121) > at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:52) > at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104) > at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to get answer that is of class com.cloud.agent.api.StartAnswer > at com.cloud.agent.manager.Commands.getAnswer(Commands.java:80) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:767) > ... 39 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira