Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3F0FC10383 for ; Wed, 20 Nov 2013 22:19:03 +0000 (UTC) Received: (qmail 18444 invoked by uid 500); 20 Nov 2013 22:19:02 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 18346 invoked by uid 500); 20 Nov 2013 22:19:02 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 18338 invoked by uid 99); 20 Nov 2013 22:19:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Nov 2013 22:19:02 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW X-Spam-Check-By: apache.org Received-SPF: unknown (athena.apache.org: error in processing during lookup of sahmed@cloudops.com) Received: from [209.85.213.48] (HELO mail-yh0-f48.google.com) (209.85.213.48) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Nov 2013 22:18:58 +0000 Received: by mail-yh0-f48.google.com with SMTP id f73so1288784yha.35 for ; Wed, 20 Nov 2013 14:18:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type; bh=dIRFDpuNDlVwvrHFzp2sHPWHoPM/T57q/SR1yFXoQnE=; b=PHZdq/4wxdIrusqkBazWlMg2NUirlHvEC9mI11A3s0SFFz+gFR6sTo5muYNilANPph fOxcvRHXL1x6gKx4MOyS9w3bzKIDspTkPFtbBvm0JkSMltK9I8M7aHIknu272/nzvZYE wKYJqqFNdnatYG9YBMccmaq4pXSdSAieyNR+YzJcvPrWkh8G414pccVhV2yv/wnSTZm9 PsuM6IMReNXTdN6jGcOZob6yNqHj3VAqgZBT5Fl5XCWyPkIVERwO4a1wZo1TZJbpI7Ql Hai/d9qYZqBi+zdMZeMWRLPj9pD4eaEHB3Skuxbr3x01iPy3VhVujRxjoNPaw+QHBfFr ALOw== X-Gm-Message-State: ALoCoQn/De6GJgvo9OTxTWMGd3SowCzWikMTVRZ6bRlZd+B532sNujXai0wj1/Kc/eXsWefPMplk X-Received: by 10.236.208.104 with SMTP id p68mr73282yho.103.1384985917239; Wed, 20 Nov 2013 14:18:37 -0800 (PST) Received: from [172.16.11.150] ([38.108.74.28]) by mx.google.com with ESMTPSA id 48sm45177354yhq.11.2013.11.20.14.18.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 20 Nov 2013 14:18:36 -0800 (PST) Message-ID: <528D3536.9080301@cloudops.com> Date: Wed, 20 Nov 2013 17:18:30 -0500 From: Syed Ahmed User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130404 Thunderbird/17.0.5 MIME-Version: 1.0 To: dev@cloudstack.apache.org CC: Wei ZHOU Subject: Re: Router requires upgrade. Unable to send command to router Error References:

<528CE091.6080606@cloudops.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------000706000104000903040200" X-Virus-Checked: Checked by ClamAV on apache.org --------------000706000104000903040200 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit OK here is how far I got debugging this. I think I am missing a small thing. I hope you guys can help. So my VM template has the correct version. root@eng-ns-dev-cs1: /export/secondary/template/tmpl/1/1 # strings f3fc75d9-0240-4c71-a3bf-fb65652e4763.vhd | grep Cloudstack Cloudstack Release* 4.2.0*Tue Nov 19 23:22:37 UTC 2013 But in the database I see the following ( table domain_router ) *************************** 4. row *************************** id: 11 element_id: 4 public_mac_address: 06:48:a8:00:00:68 public_ip_address: 172.30.91.102 public_netmask: 255.255.255.0 guest_netmask: NULL guest_ip_address: NULL is_redundant_router: 0 priority: 0 is_priority_bumpup: 0 redundant_state: UNKNOWN stop_pending: 0 role: VIRTUAL_ROUTER template_version:*Cloudstack Release 3.0 Mon Feb 6 15:10:04 PST 2012* scripts_version: 725d5e5901a62c68aed0dd3463023518 vpc_id: NULL 4 rows in set (0.00 sec) I guess this is populated from the VM that gets created. On the xen the vm is r-11. I see the following version on that VM root@r-11-VM:~# cat /etc/cloudstack-release Cloudstack Release 3.0 Mon Feb 6 15:10:04 PST 2012 This means that Xen is not picking up the template present in the secondary storage. Does Xen cache the vhd files locally to avoid coming to the secondary storage? If so, how can I disable that? Also, I was looking at UpgradeRouterTemplateCmd API which basically goes through all the VRs and reboots them. It expects that when the reboot is completed, the router should have picked up the 4.2.0 version of the template ( see line 4072 in VirtualNetworkApplianceManagerImpl.java ) I try to do the reboot manually but the template remains the same. Do you guys have any more suggestions? Thanks, -Syed On Wed 20 Nov 2013 12:55:04 PM EST, Wei ZHOU wrote: > > FYI. > > I upgraded from 2.2.14 to 4.2.1. The CPVM, SSVM and VRs are working after > running *cloudstack-sysvmadm to recreate.* > > > 2013/11/20 Syed Ahmed > >> >> +1 Same error. The secondary storage VM and the Console proxy VM seem to >> be coming up alright. I see this error only when starting the virtual >> router which is preventing me from creating any instances. >> >> >> On Wed 20 Nov 2013 11:14:47 AM EST, Will Stevens wrote: >> >>> >>> I am having the same problem. I got the latest system VMs from: >>> http://jenkins.buildacloud.org/view/master/job/build-systemvm-master/ >>> lastSuccessfulBuild/artifact/tools/appliance/dist/ >>> >>> Are these the wrong System VM Templates? If so, where should I get the >>> System VM Templates to make this work again? >>> >>> Thanks, >>> >>> Will >>> >>> >>> On Thu, Nov 7, 2013 at 7:42 PM, Alena Prokharchyk < >>> Alena.Prokharchyk@citrix.com> wrote: >>> >>> Nitin, I had the same problem, but I fixed it by uploading 4.2 system >>>> >>>> templates to my secondary storage. Make sure you have the latest too. >>>> >>>> -alena. >>>> >>>> From: Nitin Mehta >>>> >>>>> >>>>> >>>> >>>> Reply-To: "dev@cloudstack.apache.org" >>>> < >>>> dev@cloudstack.apache.org> >>>> Date: Thursday, November 7, 2013 4:16 PM >>>> To: "dev@cloudstack.apache.org" < >>>> dev@cloudstack.apache.org> >>>> Subject: Router requires upgrade. Unable to send command to router >>>> Error >>>> >>>> Unable to deploy vms in the latest master because of the commit >>>> below. Is >>>> anyone noticing this on the latest master. >>>> I checked the code and there was a commit made recently 3f5b8f which is >>>> where the exception points to. >>>> >>>> WARN [o.a.c.alerts] (CapacityChecker:ctx-4f1ef01f) alertType:: 2 // >>>> dataCenterId:: 3 // podId:: 3 // clusterId:: null // message:: System >>>> Alert: Low Available Storage in cluster c3 pod p of availability >>>> zone z3 >>>> INFO [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-1:ctx-f118d6dc) Add >>>> job-44 into job monitoring >>>> WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-26:ctx-3e786331) >>>> Detecting a change in xstoolsversion for r-6-VM >>>> ERROR [c.c.v.VirtualMachineManagerImpl] (Job-Executor-1:ctx-f118d6dc >>>> ctx-d9a00f18) Failed to start instance >>>> VM[User|VM-4d5d5db2-e5ba-4bbd-b1dc-e749ac42a74c] >>>> com.cloud.utils.exception.CloudRuntimeException: Router requires >>>> upgrade. >>>> Unable to send command to router:6 >>>> at >>>> com.cloud.network.router.VirtualNetworkApplianceManager >>>> Impl.sendCommandsToRouter(VirtualNetworkApplianceManagerImpl.java:3567) >>>> at >>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl$7.execute( >>>> VirtualNetworkApplianceManagerImpl.java:3003) >>>> at >>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules( >>>> VirtualNetworkApplianceManagerImpl.java:3848) >>>> at >>>> com.cloud.network.router.VirtualNetworkApplianceManager >>>> Impl.applyDhcpEntry(VirtualNetworkApplianceManagerImpl.java:2995) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke( >>>> NativeMethodAccessorImpl.java:39) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>> DelegatingMethodAccessorImpl.java:25) >>>> >>>> >>>> >>> >>> >> >> >> > > --------------000706000104000903040200--