cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayapal Reddy Uradi <jayapalreddy.ur...@citrix.com>
Subject Re: error logs for all VRs after upgrading ACS from 4.4.2 to 4.5.1
Date Mon, 06 Jul 2015 09:16:31 GMT
Hi Sonali,

Can you please check the MS logs for NetworkUsageCommand and answer for the errors on VR start
after upgrade.

Thanks,
Jayapal
On 06-Jul-2015, at 1:44 PM, Sonali Jadhav <sonali@servercentralen.se>
 wrote:

> Hi,
> 
> I had updated acs from 4.4.2 to 4.5.1 three days back. I successfully executed cmd "
nohup cloudstack-sysvmadm -d IPaddress -u cloud -p password -a > sysvm.log 2>&1
&"  to update VRs.
> 
> I also cross checked from ACS, version of each VR. It was 4.5.
> 
> But I was seeing these errors in management logs from few VRs. so like you said I observed
in those VR, NETWORK_STATS chain was missing. I just rebooted them and then I was able to
see that chain in iptables. So I rebooted all those VRs with error logs. And now it seems
to be ok.  
> 
> /Sonali
> 
> -----Original Message-----
> From: Jayapal Reddy Uradi [mailto:jayapalreddy.uradi@citrix.com] 
> Sent: Monday, July 6, 2015 12:45 PM
> To: <users@cloudstack.apache.org>
> Subject: Re: error logs for all VRs after upgrading ACS from 4.4.2 to 4.5.1
> 
> Hi Sonali,
> 
> Can you give me some context on how this issue is seen. I want to understand when these
chains got removed in VR.
> 
> Thanks,
> Jayapal
> 
> On 06-Jul-2015, at 12:08 PM, Sonali Jadhav <sonali@servercentralen.se> wrote:
> 
>> Sorry, error log was not for all VRs but some. 
>> 
>> I don't see NETWORK_STATS chains for those. So, I just rebooted, and then there it
was. I am suspecting VR was not updated ? weird but I was able to see version 4.5 of all VRs
in ACS. 
>> I'll keep watch if error comes again.
>> 
>> Thanks for help.
>> 
>> /Sonali
>> 
>> -----Original Message-----
>> From: Jayapal Reddy Uradi [mailto:jayapalreddy.uradi@citrix.com] 
>> Sent: Monday, July 6, 2015 11:00 AM
>> To: <users@cloudstack.apache.org>
>> Subject: Re: error logs for all VRs after upgrading ACS from 4.4.2 to 4.5.1
>> 
>> 
>> It seems VR does not have the usage related iptables chains.
>> Please check the 'iptables -L -nv' for NETWORK_STATS chains.
>> These rules supposed to be configured on VR start.
>> 
>> Thanks,
>> Jayapal
>> 
>> 
>> 
>> On 06-Jul-2015, at 10:48 AM, Sonali Jadhav <sonali@servercentralen.se<mailto:sonali@servercentralen.se>>
>> wrote:
>> 
>> Hi,
>> 
>> After I upgraded ACS from 4.4.2 to 4.5.1, I am seeing following error in management
server logs, for each and every VR,
>> 
>> 
>> 2015-07-06 07:08:20,721 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-415ba06d)
Begin cleanup expired async-jobs
>> 2015-07-06 07:08:20,733 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-415ba06d)
End cleanup expired async-jobs
>> 2015-07-06 07:08:20,811 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
External devices stats collector is running...
>> 2015-07-06 07:08:20,833 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 204 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,835 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 219 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,838 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 220 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,846 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 211 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,851 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 212 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,853 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 213 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,855 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 222 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,858 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 223 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,884 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 215 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,887 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:ctx-91d5dc5d)
Found 10 running routers.
>> 2015-07-06 07:08:20,893 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 216 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,894 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-50b5b42f)
Found 11 routers to update status.
>> 2015-07-06 07:08:20,898 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-50b5b42f)
Found 0 networks to update RvR status.
>> 2015-07-06 07:08:20,899 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-8a22f14d)
Network 217 is not configured for external networking, so skipping usage check.
>> 2015-07-06 07:08:20,910 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-284:ctx-0e4c6018)
Seq 5-6790865288121300428: Executing request
>> 2015-07-06 07:08:20,915 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-284:ctx-0e4c6018)
Executing command in VR: /opt/cloud/bin/router_proxy.sh netusage.sh 169.254.2.197 -g
>> 2015-07-06 07:08:21,041 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-60440aa6)
Snapshot scheduler.poll is being called at 2015-07-06 05:08:21 GMT
>> 2015-07-06 07:08:21,043 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-60440aa6)
Got 0 snapshots to be executed at 2015-07-06 05:08:21 GMT
>> 2015-07-06 07:08:21,337 WARN  [c.c.h.x.r.XenServer56Resource] (DirectAgent-284:ctx-0e4c6018)
Failed to get network usage stats due to
>> java.lang.NumberFormatException: For input string: "iptables"
>>              at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>              at java.lang.Long.parseLong(Long.java:441)
>>              at java.lang.Long.<init>(Long.java:702)
>>              at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetworkStats(CitrixResourceBase.java:2225)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.execute(XenServer56Resource.java:204)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:62)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
>>              at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:302)
>>              at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>              at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>              at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>              at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>              at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>>              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>>              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>              at java.lang.Thread.run(Thread.java:744)
>> 2015-07-06 07:08:21,339 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-284:ctx-0e4c6018)
Seq 5-6790865288121300428: Response Received:
>> 2015-07-06 07:08:21,339 DEBUG [c.c.a.t.Request] (DirectAgent-284:ctx-0e4c6018) Seq
5-6790865288121300428: Processing:  { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags:
10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"result":false,"details":"Exception: java.lang.NumberFormatException\nMessage:
For input string: \"iptables\"\nStack: java.lang.NumberFormatException: For input string:
\"iptables\"\n\tat java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)\n\tat
java.lang.Long.parseLong(Long.java:441)\n\tat java.lang.Long.<init>(Long.java:702)\n\tat
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetworkStats(CitrixResourceBase.java:2225)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.execute(XenServer56Resource.java:204)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:62)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)\n\tat
com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)\n\tat
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:302)\n\tat
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)\n\tat
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat
java.lang.Thread.run(Thread.java:744)\n","wait":0}}] }
>> 2015-07-06 07:08:21,339 DEBUG [c.c.a.t.Request] (RouterMonitor-1:ctx-91d5dc5d) Seq
5-6790865288121300428: Received:  { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags:
10, { NetworkUsageAnswer } }
>> 2015-07-06 07:08:21,339 DEBUG [c.c.a.m.AgentManagerImpl] (RouterMonitor-1:ctx-91d5dc5d)
Details from executing class com.cloud.agent.api.NetworkUsageCommand: Exception: java.lang.NumberFormatException
>> Message: For input string: "iptables"
>> Stack: java.lang.NumberFormatException: For input string: "iptables"
>>              at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>              at java.lang.Long.parseLong(Long.java:441)
>>              at java.lang.Long.<init>(Long.java:702)
>>              at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetworkStats(CitrixResourceBase.java:2225)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.execute(XenServer56Resource.java:204)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:62)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
>>              at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
>>              at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:302)
>>              at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>              at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>              at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>              at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>              at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>>              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>>              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>              at java.lang.Thread.run(Thread.java:744)
>> 
>> what exactly is this error?
>> 
>> /sonali
>> 
> 


Mime
View raw message