Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E56C7115B8 for ; Fri, 19 Sep 2014 01:43:46 +0000 (UTC) Received: (qmail 14034 invoked by uid 500); 19 Sep 2014 01:43:46 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 13988 invoked by uid 500); 19 Sep 2014 01:43:46 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 13962 invoked by uid 99); 19 Sep 2014 01:43:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Sep 2014 01:43:46 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mike.tutkowski@solidfire.com designates 209.85.219.46 as permitted sender) Received: from [209.85.219.46] (HELO mail-oa0-f46.google.com) (209.85.219.46) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Sep 2014 01:43:41 +0000 Received: by mail-oa0-f46.google.com with SMTP id m1so692102oag.5 for ; Thu, 18 Sep 2014 18:43:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=gVrfrnWWC3ZMFlrkfRCVBA62sGg5rcM2ioxBu2b9Xf8=; b=MOCCE54q1OMAtbSNU89QrNXO4WnCZ7r+XOs/f9F40HQo4gqj45V9brfeQEB2zpqGTb fvYSPhqVBQHHfye7KFsx7uFw0qFUrP3JWQnIQXf9RD+J1iGirnvtwaDFaSIgHGMO8jc/ y6NMcCg1FKGF4UyOF56JgmUprfPFVcVZLKGF2qvnlI0k9ntyL7e4ojOsjyVtWXB0Pv0F Twvnn6LsYrwljFAdZ4xYDCy114Ze1ViNm8dK+7bj4+Uc6NF9cr3eYC01kQvdQlDrWY4r ai0/TqN/QDrrFohg6pKDbT0yFLinRdRspeAXK5u0+c4qvf8O+7zPMEIrKSL1l/X38lDJ meiw== X-Gm-Message-State: ALoCoQm8/142GSPe3BXlmyJ06VZVxwNXaWcgsXeRNxXaVehZ8YhT18PFP/xqL3LZ/uwpROcpbSNj MIME-Version: 1.0 X-Received: by 10.182.52.165 with SMTP id u5mr3855963obo.30.1411091000988; Thu, 18 Sep 2014 18:43:20 -0700 (PDT) Received: by 10.182.24.106 with HTTP; Thu, 18 Sep 2014 18:43:20 -0700 (PDT) In-Reply-To: References: <541B782B.2020008@gmail.com> Date: Thu, 18 Sep 2014 19:43:20 -0600 Message-ID: Subject: Re: [VMWARE SETUP ERROR] Error seeing in brining up System VMs with Vmware setup with CS server From: Mike Tutkowski To: ilya musayev Cc: "dev@cloudstack.apache.org" , Koushik Das , Sateesh Chodapuneedi , Adip Shetty Content-Type: multipart/alternative; boundary=089e0158aaa6dc2d4d05036139a9 X-Virus-Checked: Checked by ClamAV on apache.org --089e0158aaa6dc2d4d05036139a9 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable OK, I checked in a fix for this: https://git-wip-us.apache.org/repos/asf?p=3Dcloudstack.git;a=3Dcommitdiff;h= =3D18e5aa116e99906f24a292bf7546809e05c72ec2;hp=3D7537c33bcdda65672fdd8f8150= 9e6885aa536932 I did notice that both before and after my fix that the virtual router is NOT coming up. The system VM being cloned and the starting of the VM looks fine. It gets to the command prompt and then a little while later is shut down and restarted. I get an error in the GUI indicating that my VM deployment failed. I don't personally know a whole lot about how the virtual router code works, so we should pass that on to an appropriate developer. On Thu, Sep 18, 2014 at 6:35 PM, Mike Tutkowski < mike.tutkowski@solidfire.com> wrote: > Just destroyed and re-created my env from scratch without > 8bb4022f3729154f85036c4a7f55e2de783e4909 and it works fine. > > I think I know what the author was intending with the code that is causin= g > the problem, so I'll go ahead and put in what I think is a fix, re-create > my env, and see if it works. > > On Thu, Sep 18, 2014 at 6:26 PM, ilya musayev > wrote: > >> Thank you Mike, please keep us posted. >> >> Regards >> ilya >> >> On 9/18/14, 4:14 PM, Mike Tutkowski wrote: >> >> Little clarification on this: >> >> "The dsName is derived from the UUID of the datastore." >> >> I mean that the dsName is derived from the UUID primary storage in the >> DB. >> >> On Thu, Sep 18, 2014 at 5:12 PM, Mike Tutkowski < >> mike.tutkowski@solidfire.com> wrote: >> >>> Just a little update on this: >>> >>> The problem doesn't seem to have anything to do with "managed" storage >>> code that was introduced back in March. >>> >>> The dsName is derived from the UUID of the datastore. This leads to a >>> null being returned from diskInfoBuilder.getDiskInfoByBackingFileBaseNa= me, >>> which subsequently leads to a null being returned from this method (and= a >>> NullPointerException being thrown). >>> >>> I plan to take a look at the history of this file in gitk. >>> >>> private VirtualMachineDiskInfo >>> getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, >>> DiskTO vol) { >>> >>> if (diskInfoBuilder !=3D null) { >>> >>> VolumeObjectTO volume =3D (VolumeObjectTO)vol.getData(); >>> >>> String dsName =3D volume.getDataStore().getUuid().replace("= -", >>> ""); >>> >>> >>> Map details =3D vol.getDetails(); >>> >>> boolean isManaged =3D details !=3D null && >>> Boolean.parseBoolean(details.get(DiskTO.MANAGED)); >>> >>> >>> VirtualMachineDiskInfo diskInfo =3D >>> >>> >>> diskInfoBuilder.getDiskInfoByBackingFileBaseName(isManaged ? new >>> DatastoreFile(volume.getPath()).getFileBaseName() : volume.getPath(), >>> dsName); >>> >>> On Thu, Sep 18, 2014 at 4:34 PM, Mike Tutkowski < >>> mike.tutkowski@solidfire.com> wrote: >>> >>>> I think I found the magic. :) >>>> >>>> sudo mount -t nfs 192.168.129.46:/export/secondary /mnt/sec >>>> >>>> If I mount /export/secondary to /mnt/sec (I did it in a bit of a >>>> roundabout way here), then the management server is happy. >>>> >>>> I think this is only VMware "thing" for the management server. >>>> >>>> Does anyone know if we document this? >>>> >>>> Thanks! >>>> >>>> On Thu, Sep 18, 2014 at 4:18 PM, Mike Tutkowski < >>>> mike.tutkowski@solidfire.com> wrote: >>>> >>>>> The instructions even say the following: >>>>> >>>>> If your secondary storage mount point is not named /mnt/secondary, >>>>> substitute your own mount point name. >>>>> >>>>> On Thu, Sep 18, 2014 at 4:15 PM, Mike Tutkowski < >>>>> mike.tutkowski@solidfire.com> wrote: >>>>> >>>>>> Yeah, I've been through the instructions a couple times and don't se= e >>>>>> anything that indicates why it's having trouble with /mnt/sec (it sa= ys >>>>>> /mnt/sec instead of /mnt/secondary in the log messages). >>>>>> >>>>>> On Thu, Sep 18, 2014 at 4:01 PM, Ritu Sabharwal >>>>> > wrote: >>>>>> >>>>>>> I have an external NFS server and it is mounted to /mnt/secondary o= n >>>>>>> my CS server so seeding command I used is: >>>>>>> >>>>>>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-instal= l-sys-tmplt >>>>>>> -m /mnt/secondary -u >>>>>>> http://cloudstack.apt-get.eu/systemvm/4.4/systemvm64template-4.4.0-= 6-vmware.ova >>>>>>> -h vmware -F >>>>>>> >>>>>>> I guess, if the secondary storage is local, then /export/secondary >>>>>>> should be fine. >>>>>>> >>>>>>> Regards, >>>>>>> Ritu S. >>>>>>> -----Original Message----- >>>>>>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com] >>>>>>> Sent: Thursday, September 18, 2014 2:34 PM >>>>>>> To: dev@cloudstack.apache.org >>>>>>> Cc: ilya musayev; Koushik Das; Sateesh Chodapuneedi; Adip Shetty >>>>>>> Subject: Re: [VMWARE SETUP ERROR] Error seeing in brining up System >>>>>>> VMs with Vmware setup with CS server >>>>>>> >>>>>>> The instructions to seed secondary storage for VMware look like thi= s: >>>>>>> >>>>>>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-instal= l-sys-tmplt >>>>>>> \ -m /mnt/secondary \ -u >>>>>>> http://cloudstack.apt-get.eu/systemvm/4.4/systemvm64template-4.4.0-= 6-vmware.ova >>>>>>> \ >>>>>>> -h vmware \ >>>>>>> -s \ -F >>>>>>> >>>>>>> For -m, I have /export/secondary (not /mnt/secondary). I assume >>>>>>> that's correct because that is, in fact, where my secondary storage= system >>>>>>> template is located for XenServer and that seems to work just fine. >>>>>>> >>>>>>> >>>>>>> On Thu, Sep 18, 2014 at 3:20 PM, Mike Tutkowski < >>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>> >>>>>>> > I'm not getting far enough along in the process to repro (or not) >>>>>>> the >>>>>>> > problem. >>>>>>> > >>>>>>> > I do have a /mnt/sec folder (and I even made it 777). >>>>>>> > >>>>>>> > Any thoughts on this? >>>>>>> > >>>>>>> > WARN [c.c.s.r.VmwareStorageProcessor] (DirectAgent-46:ctx-fdd23e= c9 >>>>>>> > 192.168.129.71, job-95/job-123, cmd: CopyCommand) Exception: tar >>>>>>> > --no-same-owner -xf /mnt/sec/template/tmpl/1/8//routing-8.ova >>>>>>> > java.io.IOException: Cannot run program "tar" (in directory >>>>>>> > "/mnt/sec/template/tmpl/1/8"): error=3D2, No such file or directo= ry >>>>>>> > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041) >>>>>>> > at com.cloud.utils.script.Script.execute(Script.java:195) >>>>>>> > at com.cloud.utils.script.Script.execute(Script.java:163) >>>>>>> > at com.cloud.storage.res >>>>>>> > >>>>>>> > On Wed, Sep 17, 2014 at 2:53 PM, Mike Tutkowski < >>>>>>> > mike.tutkowski@solidfire.com> wrote: >>>>>>> > >>>>>>> >> You're right, Marcus. Now that I think about it, asserts are >>>>>>> ignored >>>>>>> >> by default. >>>>>>> >> >>>>>>> >> On Wed, Sep 17, 2014 at 1:59 PM, Marcus >>>>>>> wrote: >>>>>>> >> >>>>>>> >>> Aren't asserts ignored by default? I think we had an issue with >>>>>>> >>> asserts in the past, and I believe at the time the default conf= ig >>>>>>> >>> had asserts disabled. >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> On Wed, Sep 17, 2014 at 1:35 PM, Ritu Sabharwal >>>>>>> >>> >>>>>>> >>> wrote: >>>>>>> >>> >>>>>>> >>> > Hi Mike, >>>>>>> >>> > >>>>>>> >>> > I updated the master just now and tried the setup. I see the >>>>>>> same >>>>>>> >>> error on >>>>>>> >>> > same line number. >>>>>>> >>> > >>>>>>> >>> > Thanks & Regards, >>>>>>> >>> > Ritu S. >>>>>>> >>> > >>>>>>> >>> > -----Original Message----- >>>>>>> >>> > From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com] >>>>>>> >>> > Sent: Tuesday, September 16, 2014 8:32 PM >>>>>>> >>> > To: dev@cloudstack.apache.org >>>>>>> >>> > Cc: ilya musayev; Koushik Das; Sateesh Chodapuneedi; Adip >>>>>>> Shetty >>>>>>> >>> > Subject: Re: [VMWARE SETUP ERROR] Error seeing in brining up >>>>>>> >>> > System VMs with Vmware setup with CS server >>>>>>> >>> > >>>>>>> >>> > Can you tell us what commit SHA you are running under? >>>>>>> >>> > >>>>>>> >>> > I just updated master and line 2094 in VmwareResource is the >>>>>>> following: >>>>>>> >>> > >>>>>>> >>> > String[] diskChain =3D diskInfo.getDiskChain(); >>>>>>> >>> > >>>>>>> >>> > The only candidate for a NullPointerException there is >>>>>>> diskInfo; >>>>>>> >>> however, >>>>>>> >>> > the previous line would have caught this as it looks like thi= s: >>>>>>> >>> > >>>>>>> >>> > assert (diskInfo !=3D null); >>>>>>> >>> > >>>>>>> >>> > That being the case, I'm curious what commit SHA you saw this >>>>>>> on? >>>>>>> >>> > >>>>>>> >>> > Thanks! >>>>>>> >>> > >>>>>>> >>> > On Tue, Sep 16, 2014 at 5:22 PM, Ritu Sabharwal >>>>>>> >>> > >>>>>>> >>> > wrote: >>>>>>> >>> > >>>>>>> >>> > > Hi Sateesh, Koushik, >>>>>>> >>> > > >>>>>>> >>> > > I am setting up Vmware Cluster with CS (master) and seeing >>>>>>> errors. >>>>>>> >>> > > ilya was helping me for this setup and figured out this >>>>>>> error. >>>>>>> >>> > > >>>>>>> >>> > > The System VMs(Secondary Stogare VM and Console Proxy VM) a= re >>>>>>> >>> > > not coming up properly. The system vms are reconfiguring an= d >>>>>>> >>> > > destroying again and again in cycle. >>>>>>> >>> > > >>>>>>> >>> > > I have used Vmware Vsphere 5.1 SDK for building CS. >>>>>>> >>> > > >>>>>>> >>> > > The error logs give this error: >>>>>>> >>> > > >>>>>>> >>> > > 2014-09-16 16:11:01,619 WARN [c.c.h.v.r.VmwareResource] >>>>>>> >>> > > (DirectAgent-8:ctx-8909215e 10.24.41.149, job-54/job-68, cm= d: >>>>>>> >>> > > StartCommand) StartCommand failed due to Exception: >>>>>>> >>> > > java.lang.NullPointerException >>>>>>> >>> > > Message: null >>>>>>> >>> > > >>>>>>> >>> > > java.lang.NullPointerException >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> com.cloud.hypervisor.vmware.resource.VmwareResource.postDiskConfigB= e >>>>>>> >>> foreStart(VmwareResource.java:2094) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareR= e >>>>>>> >>> source.java:1685) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(= V >>>>>>> >>> mwareResource.java:448) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(Direct= A >>>>>>> >>> gentAttache.java:294) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(= M >>>>>>> >>> anagedContextRunnable.java:49) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.= c >>>>>>> >>> all(DefaultManagedContext.java:56) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.ca= l >>>>>>> >>> lWithContext(DefaultManagedContext.java:103) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.ru= n >>>>>>> >>> WithContext(DefaultManagedContext.java:53) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Ma= n >>>>>>> >>> agedContextRunnable.java:46) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:= 4 >>>>>>> >>> 71) >>>>>>> >>> > > at >>>>>>> java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTas= k >>>>>>> >>> .access$201(ScheduledThreadPoolExecutor.java:178) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTas= k >>>>>>> >>> .run(ScheduledThreadPoolExecutor.java:292) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r >>>>>>> >>> .java:1145) >>>>>>> >>> > > at >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut= o >>>>>>> >>> r.java:615) >>>>>>> >>> > > at java.lang.Thread.run(Thread.java:744) >>>>>>> >>> > > >>>>>>> >>> > > Please help to resolve this issue. >>>>>>> >>> > > >>>>>>> >>> > > Thanks & Regards, >>>>>>> >>> > > Ritu S. >>>>>>> >>> > > >>>>>>> >>> > > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> >>> > -- >>>>>>> >>> > *Mike Tutkowski* >>>>>>> >>> > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>> >>> > e: mike.tutkowski@solidfire.com >>>>>>> >>> > o: 303.746.7302 >>>>>>> >>> > Advancing the way the world uses the cloud >>>>>>> >>> > *=E2=84= =A2* >>>>>>> >>> > >>>>>>> >>> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> -- >>>>>>> >> *Mike Tutkowski* >>>>>>> >> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>> >> e: mike.tutkowski@solidfire.com >>>>>>> >> o: 303.746.7302 >>>>>>> >> Advancing the way the world uses the cloud >>>>>>> >> *=E2=84=A2= * >>>>>>> >> >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > -- >>>>>>> > *Mike Tutkowski* >>>>>>> > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>> > e: mike.tutkowski@solidfire.com >>>>>>> > o: 303.746.7302 >>>>>>> > Advancing the way the world uses the cloud >>>>>>> > *=E2=84=A2* >>>>>>> > >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Mike Tutkowski* >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>> e: mike.tutkowski@solidfire.com >>>>>>> o: 303.746.7302 >>>>>>> Advancing the way the world uses the cloud >>>>>>> *=E2=84=A2* >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Tutkowski* >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>> e: mike.tutkowski@solidfire.com >>>>>> o: 303.746.7302 >>>>>> Advancing the way the world uses the cloud >>>>>> *=E2=84=A2* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Mike Tutkowski* >>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>> e: mike.tutkowski@solidfire.com >>>>> o: 303.746.7302 >>>>> Advancing the way the world uses the cloud >>>>> *=E2=84=A2* >>>>> >>>> >>>> >>>> >>>> -- >>>> *Mike Tutkowski* >>>> *Senior CloudStack Developer, SolidFire Inc.* >>>> e: mike.tutkowski@solidfire.com >>>> o: 303.746.7302 >>>> Advancing the way the world uses the cloud >>>> *=E2=84=A2* >>>> >>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkowski@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the cloud >>> *=E2=84=A2* >>> >> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkowski@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the cloud >> *=E2=84=A2* >> >> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkowski@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the cloud > *=E2=84=A2* > --=20 *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkowski@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud *=E2=84=A2* --089e0158aaa6dc2d4d05036139a9--