Return-Path: X-Original-To: apmail-incubator-cloudstack-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-cloudstack-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4D33DE2FB for ; Thu, 10 Jan 2013 09:30:15 +0000 (UTC) Received: (qmail 22328 invoked by uid 500); 10 Jan 2013 09:30:14 -0000 Delivered-To: apmail-incubator-cloudstack-dev-archive@incubator.apache.org Received: (qmail 22206 invoked by uid 500); 10 Jan 2013 09:30:14 -0000 Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cloudstack-dev@incubator.apache.org Delivered-To: mailing list cloudstack-dev@incubator.apache.org Received: (qmail 22171 invoked by uid 99); 10 Jan 2013 09:30:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jan 2013 09:30:13 +0000 Date: Thu, 10 Jan 2013 09:30:13 +0000 (UTC) From: "Kishan Kavala (JIRA)" To: cloudstack-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CLOUDSTACK-721) Bytes sent/received in user statistics is empty (CloudStack 4.0) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-721?page=3Dcom.atlas= sian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D= 13549475#comment-13549475 ]=20 Kishan Kavala commented on CLOUDSTACK-721: ------------------------------------------ commit cd7f7716155c3cd2d29b97d7ea302518d81fdfab Author: Kishan Kavala Date: Thu Jan 10 14:53:29 2013 +0530 CLOUDSTACK-721: Fixed network usage. Send network usage command for iso= lated guest nic of non VPC VR. Send network usage command for public nic in= VPC VR. =20 > Bytes sent/received in user statistics is empty (CloudStack 4.0) > ---------------------------------------------------------------- > > Key: CLOUDSTACK-721 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-721 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the defa= ult.)=20 > Components: Usage > Affects Versions: 4.0.0 > Reporter: Ilia Shakitko > Assignee: Kishan Kavala > Priority: Critical > Labels: network, usage, user_statistics > Fix For: 4.1.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > The usage server can give the system usage of each virtual machine (such = as running time, ServiceOffering, IPAddress, Volume, Template, ISO, Port Fo= rwarding Rule, Network offering), except the network bytes sent/received. > This problem only exists in CloudStack 4.0. In CloudStack 3.0.2, the us= age server works well.=20 > Analysis steps: > 1. Check the CloudStack logs > We can get the logs in /var/log/cloud/management/management-server.log > 2012-12-18 15:05:51,130 DEBUG [agent.transport.Request] (AgentManager-Han= dler-8:null) Seq 1-1158742136: Processing: { Ans: , MgmtId: 345051509349, = via: 1, Ver: v1, Flags: 10, [{"NetworkUsageAnswer":{"routerName":"r-4-VM","= bytesSent":5928,"bytesReceived":6188,"result":true,"details":"","wait":0}}]= } > 2012-12-18 15:05:51,130 DEBUG [agent.transport.Request] (RouterMonitor-1:= null) Seq 1-1158742136: Received: { Ans: , MgmtId: 345051509349, via: 1, V= er: v1, Flags: 10, { NetworkUsageAnswer } } > 2012-12-18 15:05:51,130 DEBUG [agent.manager.AgentManagerImpl] (RouterMon= itor-1:null) Details from executing class com.cloud.agent.api.NetworkUsageC= ommand: > 2012-12-18 15:05:51,131 WARN [network.router.VirtualNetworkApplianceMana= gerImpl] (RouterMonitor-1:null) unable to find stats for account: 2 > We can see that AgentManager works well. It can get the network usage. Ho= wever, there is a problem in subsequent data processing. > =20 > 2. Check the source code and the value in database.=20 > We can see the data processing procedure in /cloud-server/src/com/cloud/n= etwork/router/VirtualNetworkApplianceManagerImpl.java (I have deleted some = unrelated source codes) > public void run() { > for (DomainRouterVO router : routers) { > String privateIP =3D router.getPrivateIpAddress(); = // from domain_router table (id= =3D4, name=3Dr-4-VM, vm_instance.private_ip_address=3DPRIVATE_IP_ADDRESS) > =20 > if (privateIP !=3D null) { > List routerNics =3D _nicDao.listByVmId(router.getId()); = // from nics table (instance_id=3D4). Found= 3 records. Return (networkid=3D 204/202/200) > for (Nic routerNic : routerNics) { > Network network =3D _networkMgr.getNetwork(routerNic.getNetworkId());= // from nics table. > if (network.getTrafficType() =3D=3D TrafficType.Public) { = // get traffic_type in network= s table. Return Guest/Control/Public. > UserStatisticsVO previousStats =3D _statsDao.findBy(router.getAcc= ountId(),=20 > router.getDataCenterIdToDeployIn(), network.getId(), null= , router.getId(), router.getType().toString()); > =20 > if (answer !=3D null) { > Transaction txn =3D Transaction.open(Transaction.CLOUD_DB); > try { > txn.start(); > UserStatisticsVO stats =3D _statsDao.lock(router.getAccou= ntId(),=20 > router.getDataCenterIdToDeployIn(), network.getId= (), routerNic.getIp4Address(), router.getId(), router.getType().toString())= ; // the parameter is =E2=80=9C2, 1, 200, PUBLIC_IP_ADDRESS, 4=E2=80= =9D. It=E2=80=99s inconsistent to database. > if (stats =3D=3D null) { > s_logger.warn("unable to find stats for account: " + = router.getAccountId()); > continue; > } > =E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6.= . > } > We can see the data initialization in /cloud-server/src/com/cloud/vm/dao/= DomainRouterDaoImpl.java > public void addRouterToGuestNetwork(VirtualRouter router, Network guestNe= twork) { > if (stats =3D=3D null) { > stats =3D new UserStatisticsVO(router.getAccountId(), router.getD= ataCenterIdToDeployIn(), null, router.getId(), = // the parameter is =E2=80=9C2, 1, null, 204, 4=E2=80=9D. It=E2= =80=99s consistent to database. > router.getType().toString(), guestNetwork.getId()); > _userStatsDao.persist(stats); > } > } > 3. Check the database. We can see the data is same to data in data initia= lization in Step (2), but different to data in data processing. > mysql> select id,data_center_id,account_id,public_ip_address,device_id,de= vice_type,network_id from user_statistics; > +----+----------------+------------+-------------------+-----------+-----= ---------+------------+ > | id | data_center_id | account_id | public_ip_address | device_id | devi= ce_type | network_id | > +----+----------------+------------+-------------------+-----------+-----= ---------+------------+ > | 1 | 1 | 2 | NULL | = 4 | DomainRouter | 204 | > | 2 | 1 | 2 | NULL | = 6 | DomainRouter | 205 | > =20 > 4. Testing. If we insert a record into the databases. The record is same = to data processing. It will work! > | 3 | 1 | 2 | PUBLIC_IP_ADDRESS | 4 = | DomainRouter | 200 | > ... after 5 minutes, we can get the VR data traffic.It looks like > | id | data_center_id | account_id | public_ip_address | device_id | devi= ce_type | network_id | net_bytes_received | net_bytes_sent | current_byt= es_received | current_bytes_sent | agg_bytes_received | agg_bytes_sent | > | 3 | 1 | 2 | PUBLIC_IP_ADDRESS | 4= | DomainRouter | 200 | 0 | = 0 | 2284239008 | 33613977 | = 2284239008 | 33613977 | > =20 > From Step 1 to Step 4, we can conclude that the inconsistent of data init= ialization and data processing procedure is the root cause of this problem. > -------------------------------------------------------------------------= ---------------------------------------------------------------------------= ------------------------------------------------ > We notice that there are some changes of the data processing procedure be= tween CloudStack 3.0 and CloudStack 4.0. That's why this problem only exist= s in CloudStack 4.0.0 > Does anybody know why this change was made or it's just a bug? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs For more information on JIRA, see: http://www.atlassian.com/software/jira