Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CD625F554 for ; Tue, 13 Aug 2013 10:04:50 +0000 (UTC) Received: (qmail 92982 invoked by uid 500); 13 Aug 2013 10:04:50 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 92951 invoked by uid 500); 13 Aug 2013 10:04:50 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 92944 invoked by uid 500); 13 Aug 2013 10:04:50 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 92941 invoked by uid 99); 13 Aug 2013 10:04:50 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Aug 2013 10:04:50 +0000 Date: Tue, 13 Aug 2013 10:04:49 +0000 (UTC) From: "Wido den Hollander (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CLOUDSTACK-4217) ceph:addprimary storage is failed with exception(documentation is not clear) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-4217?page=3Dcom.atla= ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId= =3D13738052#comment-13738052 ]=20 Wido den Hollander commented on CLOUDSTACK-4217: ------------------------------------------------ client.admin should be the correct user, so I'm not sure about why this fai= led. "ceph" isn't a valid cephx user, it should be either "client.ceph" or "clie= nt.admin" Does this Ceph cluster have cephx enabled? =20 > ceph:addprimary storage is failed with exception(documentation is not cle= ar) > -------------------------------------------------------------------------= --- > > Key: CLOUDSTACK-4217 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-421= 7 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the defa= ult.)=20 > Components: Doc, Management Server > Affects Versions: 4.2.0 > Reporter: sadhu suresh > Assignee: Wido den Hollander > Priority: Minor > > Steps: > 1.created and configured the ceph cluster. > 2. Install cloudstack agent on ubuntu13.4 and add it in the cloud-stack'= s kVM cluster > 3. Install librdos package on kvm host and also install ceph-common pakag= e > 4. able to create the image from kvm host using qemu-img command > root@Rack1Pod1Host21:/etc/ceph# qemu-img create -f rbd rbd:data/foo 1G > Formatting 'rbd:data/foo', fmt=3Drbd size=3D1073741824 cluster_size=3D0 > 5.try to create a primay storage by selecting the rbd as protocol and pr= ovinding cpeh cluster details > Actual result: > Addprimary storage "failed with delete storagepool failed error" and log= shows intially createstoragepool command successful and fail to access the= storage pool. > Agent.log: > :{"_command":{"hostType":"Routing","hostId":7,"wait":0},"result":true,"wa= it":0}}] } > 2013-08-09 22:13:16,905 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3= :null) Processing command: com.cloud.agent.api.GetHostStatsCommand > 2013-08-09 22:13:16,905 DEBUG [kvm.resource.LibvirtComputingResource] (ag= entRequest-Handler-3:null) Executing: /bin/bash -c idle=3D$(top -b -n 1|gre= p Cpu\(s\):|cut -d% -f4|cut -d, -f2);echo $idle > 2013-08-09 22:13:17,069 DEBUG [kvm.resource.LibvirtComputingResource] (ag= entRequest-Handler-3:null) Execution is successful. > 2013-08-09 22:13:17,070 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3= :null) Seq 7-1066205442: { Ans: , MgmtId: 7296881000534, via: 7, Ver: v1, = Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"empty = String","wait":0}}] } > 2013-08-09 22:13:52,969 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4= :null) Request:Seq 7-1066205443: { Cmd , MgmtId: 7296881000534, via: 7, Ve= r: v1, Flags: 100011, [{"com.cloud.agent.api.CreateStoragePoolCommand":{"ad= d":true,"pool":{"id":19,"uuid":"2cd21530-1fb9-3aff-95bc-ba3b3b3f2b12","host= ":"10.147.41.3","path":"data","userInfo":"ceph:QVFCVjdBRlNzTG1aRUJBQW1HaXZB= ay9CVjFpTXZjYVJIUWZpWGc9PQ=3D=3D","port":6789,"type":"RBD"},"localPath":"/m= nt//ed2dce31-1512-366e-84a8-a296cc5768f0","wait":0}}] } > 2013-08-09 22:13:52,969 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4= :null) Processing command: com.cloud.agent.api.CreateStoragePoolCommand > 2013-08-09 22:13:52,969 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4= :null) Seq 7-1066205443: { Ans: , MgmtId: 7296881000534, via: 7, Ver: v1, = Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"success= ","wait":0}}] } > 2013-08-09 22:13:53,019 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5= :null) Request:Seq 7-1066205444: { Cmd , MgmtId: 7296881000534, via: 7, Ve= r: v1, Flags: 100011, [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"ad= d":true,"pool":{"id":19,"uuid":"2cd21530-1fb9-3aff-95bc-ba3b3b3f2b12","host= ":"10.147.41.3","path":"data","userInfo":"ceph:QVFCVjdBRlNzTG1aRUJBQW1HaXZB= ay9CVjFpTXZjYVJIUWZpWGc9PQ=3D=3D","port":6789,"type":"RBD"},"localPath":"/m= nt//ed2dce31-1512-366e-84a8-a296cc5768f0","wait":0}}] } > 2013-08-09 22:13:53,020 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5= :null) Processing command: com.cloud.agent.api.ModifyStoragePoolCommand > 2013-08-09 22:13:53,022 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) createStoragePool didn't find existing running pool:= org.libvirt.LibvirtException: Storage pool not found: no storage pool with= matching uuid ,=C3=AF=C2=BF=C2=BD^U0^_=C3=AF=C2=BF=C2=BD:=C3=AF=C2=BF=C2= =BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD;;?+^R, need to cr= eate it > 2013-08-09 22:13:53,022 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) Didn't find an existing storage pool 2cd21530-1fb9-3= aff-95bc-ba3b3b3f2b12 by UUID, checking for pools with duplicate paths > 2013-08-09 22:13:53,025 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) Checking path of existing pool 99e0dae1-1ac8-46fa-a2= 3e-20938b543c25 against pool we want to create > 2013-08-09 22:13:53,029 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) Attempting to create storage pool 2cd21530-1fb9-3aff= -95bc-ba3b3b3f2b12 > 2013-08-09 22:13:53,029 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) > 2cd21530-1fb9-3aff-95bc-ba3b3b3f2b12 > > ceph@10.147.41.3:6789/data > > > 2013-08-09 22:13:53,031 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) org.libvirt.LibvirtException: internal error a secre= t with UUID 2cd21530-1fb9-3aff-95bc-ba3b3b3f2b12 is already defined for use= with client.admin@10.147.41.3:6789/data > 2013-08-09 22:13:53,031 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) > 2cd21530-1fb9-3aff-95bc-ba3b3b3f2b12 > 2cd21530-1fb9-3aff-95bc-ba3b3b3f2b12 > > > data > > > > > > 2013-08-09 22:13:53,036 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) org.libvirt.LibvirtException: An error occurred, but= the cause is unknown > 2013-08-09 22:13:53,037 WARN [cloud.agent.Agent] (agentRequest-Handler-5= :null) Caught: > java.lang.NullPointerException > at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createS= toragePool(LibvirtStorageAdaptor.java:526) > at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createS= toragePool(KVMStoragePoolManager.java:112) > at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createS= toragePool(KVMStoragePoolManager.java:105) > at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.exe= cute(LibvirtComputingResource.java:2572) > at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.exe= cuteRequest(LibvirtComputingResource.java:1242) > at com.cloud.agent.Agent.processRequest(Agent.java:525) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:85= 2) > at com.cloud.utils.nio.Task.run(Task.java:83) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1146) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > 2013-08-09 22:13:53,038 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5= :null) Seq 7-1066205444: { Ans: , MgmtId: 7296881000534, via: 7, Ver: v1, = Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"java.l= ang.NullPointerException\n\tat com.cloud.hypervisor.kvm.storage.LibvirtStor= ageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:526)\n\tat com.clou= d.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStorage= PoolManager.java:112)\n\tat com.cloud.hypervisor.kvm.storage.KVMStoragePool= Manager.createStoragePool(KVMStoragePoolManager.java:105)\n\tat com.cloud.h= ypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingRes= ource.java:2572)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingRe= source.executeRequest(LibvirtComputingResource.java:1242)\n\tat com.cloud.a= gent.Agent.processRequest(Agent.java:525)\n\tat com.cloud.agent.Agent$Agent= RequestHandler.doTask(Agent.java:852)\n\tat com.cloud.utils.nio.Task.run(Ta= sk.java:83)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadP= oolExecutor.java:1146)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker= .run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:67= 9)\n","wait":0}}] } > 2013-08-09 22:14:14,873 DEBUG [kvm.resource.LibvirtComputingResource] (Ug= entTask-5:null) Executing: /usr/share/cloudstack-common/scripts/vm/network/= security_group.py get_rule_logs_for_vms > 2013-08-09 22:14:14,946 DEBUG [kvm.resource.LibvirtComputingResource] (Ug= entTask-5:null) > = =20 > Host:ubuntu13.4 > virsh # version > Compiled against library: libvirt 1.0.2 > Using library: libvirt 1.0.2 > Using API: QEMU 1.0.2 > Running hypervisor: QEMU 1.4.0 > management.log: > cut -d, -f2);echo $idle > 2013-08-09 18:11:14,696 DEBUG [kvm.resource.LibvirtComputingResource] (ag= entRequest-Handler-3:null) Execution is successful. > 2013-08-09 18:11:14,698 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3= :null) Seq 7-1066205192: { Ans: , MgmtId: 7296881000534, via: 7, Ver: v1, = Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"empty = String","wait":0}}] } > 2013-08-09 18:11:14,822 DEBUG [kvm.resource.LibvirtComputingResource] (Ug= entTask-3:null) Executing: /usr/share/cloudstack-common/scripts/vm/network/= security_group.py get_rule_logs_for_vms > 2013-08-09 18:11:14,892 DEBUG [kvm.resource.LibvirtComputingResource] (Ug= entTask-3:null) Execution is successful. > 2013-08-09 18:11:14,893 DEBUG [cloud.agent.Agent] (UgentTask-3:null) Send= ing ping: Seq 7-3: { Cmd , MgmtId: -1, via: 7, Ver: v1, Flags: 11, [{"com.= cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"newSt= ates":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routi= ng","hostId":7,"wait":0}}] } > 2013-08-09 18:11:14,904 DEBUG [cloud.agent.Agent] (Agent-Handler-5:null) = Received response: Seq 7-3: { Ans: , MgmtId: 7296881000534, via: 7, Ver: v= 1, Flags: 100010, [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType= ":"Routing","hostId":7,"wait":0},"result":true,"wait":0}}] } > 2013-08-09 18:11:20,624 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4= :null) Request:Seq 7-1066205193: { Cmd , MgmtId: 7296881000534, via: 7, Ve= r: v1, Flags: 100011, [{"com.cloud.agent.api.CreateStoragePoolCommand":{"ad= d":true,"pool":{"id":13,"uuid":"2a5284e1-d490-34f1-a591-e3c8ea4b8f60","host= ":"10.147.41.3","path":"rbd","userInfo":"client.admin:AQBV7AFSsLmZEBAAmGivA= k_BV1iMvcaRHQfiXg=3D=3D","port":6789,"type":"RBD"},"localPath":"/mnt//81fb9= a4e-e62f-3956-8164-f4b557bd08a6","wait":0}}] } > 2013-08-09 18:11:20,625 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4= :null) Processing command: com.cloud.agent.api.CreateStoragePoolCommand > 2013-08-09 18:11:20,625 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4= :null) Seq 7-1066205193: { Ans: , MgmtId: 7296881000534, via: 7, Ver: v1, = Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"success= ","wait":0}}] } > 2013-08-09 18:11:20,721 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5= :null) Request:Seq 7-1066205194: { Cmd , MgmtId: 7296881000534, via: 7, Ve= r: v1, Flags: 100011, [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"ad= d":true,"pool":{"id":13,"uuid":"2a5284e1-d490-34f1-a591-e3c8ea4b8f60","host= ":"10.147.41.3","path":"rbd","userInfo":"client.admin:AQBV7AFSsLmZEBAAmGivA= k_BV1iMvcaRHQfiXg=3D=3D","port":6789,"type":"RBD"},"localPath":"/mnt//81fb9= a4e-e62f-3956-8164-f4b557bd08a6","wait":0}}] } > 2013-08-09 18:11:20,721 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5= :null) Processing command: com.cloud.agent.api.ModifyStoragePoolCommand > 2013-08-09 18:11:20,723 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) createStoragePool didn't find existing running pool:= org.libvirt.LibvirtException: Storage pool not found: no storage pool with= matching uuid *R=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=944=C3=AF=C2=BF=C2= =BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BDK=C3=AF=C2=BF=C2= =BD`, need to create it > 2013-08-09 18:11:20,723 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) Didn't find an existing storage pool 2a5284e1-d490-3= 4f1-a591-e3c8ea4b8f60 by UUID, checking for pools with duplicate paths > 2013-08-09 18:11:20,726 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) Checking path of existing pool 99e0dae1-1ac8-46fa-a2= 3e-20938b543c25 against pool we want to create > 2013-08-09 18:11:20,730 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) Attempting to create storage pool 2a5284e1-d490-34f1= -a591-e3c8ea4b8f60 > 2013-08-09 18:11:20,731 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) > 2a5284e1-d490-34f1-a591-e3c8ea4b8f60 > 2a5284e1-d490-34f1-a591-e3c8ea4b8f60 > > client.admin@10.147.41.3:6789/rbd > > > 2013-08-09 18:11:20,741 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) > 2a5284e1-d490-34f1-a591-e3c8ea4b8f60 > 2a5284e1-d490-34f1-a591-e3c8ea4b8f60 > > > rbd > > > > > > 2013-08-09 18:11:20,748 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentR= equest-Handler-5:null) org.libvirt.LibvirtException: An error occurred, but= the cause is unknown > 2013-08-09 18:11:20,749 WARN [cloud.agent.Agent] (agentRequest-Handler-5= :null) Caught: > java.lang.NullPointerException > at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createS= toragePool(LibvirtStorageAdaptor.java:526) > at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createS= toragePool(KVMStoragePoolManager.java:112) > at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createS= toragePool(KVMStoragePoolManager.java:105) > at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.exe= cute(LibvirtComputingResource.java:2572) > at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.exe= cuteRequest(LibvirtComputingResource.java:1242) > at com.cloud.agent.Agent.processRequest(Agent.java:525) > at com.cloud.agent.Agent$AgentRequestHandler.doTas > = =20 > root@Rack1Pod1Host21:~# virsh pool-define ceph.xml=20 > Pool 2a5284e1-d490-34f1-a591-e3c8ea4b8f60 defined from ceph.xml -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs For more information on JIRA, see: http://www.atlassian.com/software/jira