cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sonali Jadhav <son...@servercentralen.se>
Subject RE: error logs for all VRs after upgrading ACS from 4.4.2 to 4.5.1
Date Mon, 06 Jul 2015 08:14:34 GMT
Hi,

I had updated acs from 4.4.2 to 4.5.1 three days back. I successfully executed cmd " nohup
cloudstack-sysvmadm -d IPaddress -u cloud -p password -a > sysvm.log 2>&1 &"
 to update VRs.

I also cross checked from ACS, version of each VR. It was 4.5.

But I was seeing these errors in management logs from few VRs. so like you said I observed
in those VR, NETWORK_STATS chain was missing. I just rebooted them and then I was able to
see that chain in iptables. So I rebooted all those VRs with error logs. And now it seems
to be ok.  

/Sonali

-----Original Message-----
From: Jayapal Reddy Uradi [mailto:jayapalreddy.uradi@citrix.com] 
Sent: Monday, July 6, 2015 12:45 PM
To: <users@cloudstack.apache.org>
Subject: Re: error logs for all VRs after upgrading ACS from 4.4.2 to 4.5.1

Hi Sonali,

Can you give me some context on how this issue is seen. I want to understand when these chains
got removed in VR.

Thanks,
Jayapal

On 06-Jul-2015, at 12:08 PM, Sonali Jadhav <sonali@servercentralen.se> wrote:

> Sorry, error log was not for all VRs but some. 
> 
> I don't see NETWORK_STATS chains for those. So, I just rebooted, and then there it was.
I am suspecting VR was not updated ? weird but I was able to see version 4.5 of all VRs in
ACS. 
> I'll keep watch if error comes again.
> 
> Thanks for help.
> 
> /Sonali
> 
> -----Original Message-----
> From: Jayapal Reddy Uradi [mailto:jayapalreddy.uradi@citrix.com] 
> Sent: Monday, July 6, 2015 11:00 AM
> To: <users@cloudstack.apache.org>
> Subject: Re: error logs for all VRs after upgrading ACS from 4.4.2 to 4.5.1
> 
> 
> It seems VR does not have the usage related iptables chains.
> Please check the 'iptables -L -nv' for NETWORK_STATS chains.
> These rules supposed to be configured on VR start.
> 
> Thanks,
> Jayapal
> 
> 
> 
> On 06-Jul-2015, at 10:48 AM, Sonali Jadhav <sonali@servercentralen.se<mailto:sonali@servercentralen.se>>
> wrote:
> 
> Hi,
> 
> After I upgraded ACS from 4.4.2 to 4.5.1, I am seeing following error in management server
logs, for each and every VR,
> 
> 
> 2015-07-06 07:08:20,721 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-415ba06d)
Begin cleanup expired async-jobs
> 2015-07-06 07:08:20,733 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-415ba06d)
End cleanup expired async-jobs
> 2015-07-06 07:08:20,811 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
External devices stats collector is running...
> 2015-07-06 07:08:20,833 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 204 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,835 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 219 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,838 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 220 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,846 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 211 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,851 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 212 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,853 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 213 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,855 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 222 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,858 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 223 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,884 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 215 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,887 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:ctx-91d5dc5d)
Found 10 running routers.
> 2015-07-06 07:08:20,893 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 216 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,894 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-50b5b42f)
Found 11 routers to update status.
> 2015-07-06 07:08:20,898 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-50b5b42f)
Found 0 networks to update RvR status.
> 2015-07-06 07:08:20,899 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 217 is not configured for external networking, so skipping usage check.
> 2015-07-06 07:08:20,910 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-284:ctx-0e4c6018)
Seq 5-6790865288121300428: Executing request
> 2015-07-06 07:08:20,915 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-284:ctx-0e4c6018)
Executing command in VR: /opt/cloud/bin/router_proxy.sh netusage.sh 169.254.2.197 -g
> 2015-07-06 07:08:21,041 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-60440aa6)
Snapshot scheduler.poll is being called at 2015-07-06 05:08:21 GMT
> 2015-07-06 07:08:21,043 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-60440aa6)
Got 0 snapshots to be executed at 2015-07-06 05:08:21 GMT
> 2015-07-06 07:08:21,337 WARN  [c.c.h.x.r.XenServer56Resource] (DirectAgent-284:ctx-0e4c6018)
Failed to get network usage stats due to
> java.lang.NumberFormatException: For input string: "iptables"
>               at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>               at java.lang.Long.parseLong(Long.java:441)
>               at java.lang.Long.<init>(Long.java:702)
>               at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetworkStats(CitrixResourceBase.java:2225)
>               at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.execute(XenServer56Resource.java:204)
>               at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:62)
>               at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
>               at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
>               at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:302)
>               at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>               at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>               at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>               at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>               at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>               at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>               at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>               at java.lang.Thread.run(Thread.java:744)
> 2015-07-06 07:08:21,339 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-284:ctx-0e4c6018)
Seq 5-6790865288121300428: Response Received:
> 2015-07-06 07:08:21,339 DEBUG [c.c.a.t.Request] (DirectAgent-284:ctx-0e4c6018) Seq 5-6790865288121300428:
Processing:  { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"result":false,"details":"Exception:
java.lang.NumberFormatException\nMessage: For input string: \"iptables\"\nStack: java.lang.NumberFormatException:
For input string: \"iptables\"\n\tat java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)\n\tat
java.lang.Long.parseLong(Long.java:441)\n\tat java.lang.Long.<init>(Long.java:702)\n\tat
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetworkStats(CitrixResourceBase.java:2225)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.execute(XenServer56Resource.java:204)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:62)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)\n\tat
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:302)\n\tat
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)\n\tat
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat
java.lang.Thread.run(Thread.java:744)\n","wait":0}}] }
> 2015-07-06 07:08:21,339 DEBUG [c.c.a.t.Request] (RouterMonitor-1:ctx-91d5dc5d) Seq 5-6790865288121300428:
Received:  { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags: 10, { NetworkUsageAnswer
} }
> 2015-07-06 07:08:21,339 DEBUG [c.c.a.m.AgentManagerImpl] (RouterMonitor-1:ctx-91d5dc5d)
Details from executing class com.cloud.agent.api.NetworkUsageCommand: Exception: java.lang.NumberFormatException
> Message: For input string: "iptables"
> Stack: java.lang.NumberFormatException: For input string: "iptables"
>               at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>               at java.lang.Long.parseLong(Long.java:441)
>               at java.lang.Long.<init>(Long.java:702)
>               at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetworkStats(CitrixResourceBase.java:2225)
>               at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.execute(XenServer56Resource.java:204)
>               at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:62)
>               at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
>               at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
>               at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:302)
>               at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>               at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>               at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>               at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>               at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>               at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>               at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>               at java.lang.Thread.run(Thread.java:744)
> 
> what exactly is this error?
> 
> /sonali
> 


Mime
View raw message