Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 752AFED7C for ; Thu, 17 Jan 2013 21:47:22 +0000 (UTC) Received: (qmail 36845 invoked by uid 500); 17 Jan 2013 21:47:22 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 36696 invoked by uid 500); 17 Jan 2013 21:47:22 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 36688 invoked by uid 99); 17 Jan 2013 21:47:22 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Jan 2013 21:47:22 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [202.147.45.60] (HELO mx4.optiver.com.au) (202.147.45.60) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Jan 2013 21:47:17 +0000 Received: from localhost (mx4 [127.0.0.1]) by mx4.optiver.com.au (Postfix) with ESMTP id 7DD6CBFD0E1; Fri, 18 Jan 2013 08:44:32 +1100 (EST) X-Virus-Scanned: amavisd-new at optiver.com.au Received: from mx4.optiver.com.au ([127.0.0.1]) by localhost (mx4.optiver.com.au [127.0.0.1]) (amavisd-new, port 10024) with LMTP id eLBNYH1OcgFf; Fri, 18 Jan 2013 08:44:26 +1100 (EST) Received: from mail.comp.optiver.com (opsywnvm0072.comp.optiver.com [10.70.144.41]) by mx4.optiver.com.au (Postfix) with ESMTP id 11313BFD0C0 for ; Fri, 18 Jan 2013 08:44:26 +1100 (EST) Received: from OPSYWNSR0152.comp.optiver.com ([10.70.144.40]) by opsywnvm0072.comp.optiver.com ([10.70.144.41]) with mapi; Fri, 18 Jan 2013 08:46:49 +1100 From: James Stewart To: "user@flume.apache.org" Date: Fri, 18 Jan 2013 08:46:48 +1100 Subject: RE: Constant Traffic on port 35872 Thread-Topic: Constant Traffic on port 35872 Thread-Index: Ac30nfOPYI1UCJJZSeW17XkPTfaFOQAXdB0A Message-ID: <3B7D6CD11AEFB4488B8F2E3E60E9F6DAFC63879889@OPSYWNSR0152.comp.optiver.com> References: <3B7D6CD11AEFB4488B8F2E3E60E9F6DAFC63879725@OPSYWNSR0152.comp.optiver.com> <3B7D6CD11AEFB4488B8F2E3E60E9F6DAFC63879743@OPSYWNSR0152.comp.optiver.com> <3B7D6CD11AEFB4488B8F2E3E60E9F6DAFC638797B4@OPSYWNSR0152.comp.optiver.com> <9D5BDAB1-6BCE-40FC-8E69-85BDD8D44365@gmail.com> <50F7D309.2040809@gruter.com> In-Reply-To: <50F7D309.2040809@gruter.com> Accept-Language: en-US, en-AU Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-AU Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org That was it! I added flume.reporter.poller.period in flume-conf.xml with a value of 6000= 0 and the reporting traffic is now back to a sensible level...:-). Thanks so much for your replies, they really helped me out. -----Original Message----- From: Jeong-shik Jang [mailto:jsjang@gruter.com]=20 Sent: Thursday, 17 January 2013 9:32 PM To: user@flume.apache.org Subject: Re: Constant Traffic on port 35872 You may want to check property named flume.reporter.poller.period in flume-= conf.xml or flume-site.xml; default value is 2000 (millis) in case of not b= eing defined. if you cannot find the property in flume-conf.xml, add it to flume-site.xml= and set value to 60000 (60 sec). I am not sure which version of flume-og you are using, but in what I am usi= ng (v0.9.4), the related class seems to have been deprecated. - JS On 1/17/13 4:00 PM, Alexander Alten-Lorenz wrote: > Depends on the architecture, since the nodes are configured per master's = webui. The master calls regularly the in-memory config and spread them arou= nd. This is needed for HA, as example. > Flume 1.x up has another architecture. > > - Alex > > On Jan 17, 2013, at 5:02 AM, James Stewart = wrote: > >> Yeah, I've just realised that it's*exactly* the same data that is return= ed when you connect to http://my.flume.node:35862, (for monitoring etc). Ev= en the order in which the metrics are sent is the same. >> >> So it seems that the node is generating this configuration data and pump= ing it back to the master every 1-2 seconds. This produces ~40-80Kb/sec of = largely unnecessary traffic per node, which soon adds up over a WAN. >> >> I can understand why this config data would be sent back to the master o= ccasionally but I don't understand why it does so every 1-2 seconds, ignori= ng flume.config.heartbeat.period. >> >> From: Mike Percy [mailto:mpercy@apache.org] >> Sent: Thursday, 17 January 2013 2:37 PM >> To: user@flume.apache.org >> Subject: Re: Constant Traffic on port 35872 >> >> I doubt it's the Thrift RPC layer. It's most likely the app. >> >> On Wed, Jan 16, 2013 at 5:53 PM, James Stewart > wrote: >> I thought it was only 'heartbeats' that were supposed to go via port 358= 72, so I reduced flume.config.heartbeat.period on the nodes to 60 sec. Acco= rding to the master it's only seeing heartbeats every 60 sec now, and yet I= still get constantly spammed with data on port 35872 from every node. >> >> It does look like metric collection or config reporting of some kind, li= ke it's reporting the configuration of the sources/sinks and even data abou= t the JVM: >> >> ............rt.starttime....Thu Jan 17 11:47:07 EST 2013... rt.vmnam= e...!Java HotSpot(TM) 64-Bit Server VM....name...(pn-opsynxsr0202.aus.optiv= er.com.jvm-Info....rt.vmversion....16.3-b01....rt.vmvendor....Sun Microsyst= ems Inc. >> ..........max.....@......mem.other.used................mem.other.committ= ed.....s......mem.heap.max........... >> >> But it's just the same data over and over again every second. This traff= ic is travelling across a WAN and with a lot of nodes it's a significant en= ough amount of data to be a problem. >> >> I don't know much about Java, but could this be something to do with Thr= ift? >> >> >> From: Mike Percy [mailto:mpercy@apache.org] >> Sent: Thursday, 17 January 2013 12:19 PM >> To: user@flume.apache.org >> Subject: Re: Constant Traffic on port 35872 >> >> I know next to nothing about Flume OG but if I had to guess I'd say it's= either a heartbeat or metrics collection. Why do you want it to stop? >> >> On Wed, Jan 16, 2013 at 5:06 PM, James Stewart > wrote: >> Hello all, >> >> I'm using flume 0.9.4 - before anybody mentions it, we aren't in a posit= ion to upgrade at the moment due to custom decorators + sinks. >> >> I'm seeing constant traffic from my various flume nodes back to my maste= r on port 35872. Even after increasing my timeout period to 60 sec and disa= bling all custom sources/sinks/decorators, I am still constantly receiving = packets from all of my nodes back to my master. I have included a dump of t= he tcp packets below - I receive this same traffic from every node every 1-= 2 sec. >> >> Does anybody have any idea what is generating this traffic and how I can= stop it? >> >> Many thanks, >> >> James >> >> >> V..?..-MLogicalNodeManager.my_logical_node_apps.sourceConfig....null...1= LogicalNodeManager.my_logical_node_apps.physicalnode....my_physical_node.my= _domain.com...:LogicalNodeManager.my_physical_node.my_domain.com.sinkConfig= ....null...-LogicalNodeManager.my_logical_node_apps.hostname....my_physical= _node.my_domain.com....AgentWALAckManager.name....AgentWALAckManager...8Log= icalNodeManager.my_physical_node.my_domain.com.nodename....my_physical_node= .my_domain.com...> 11:58:22.630881 IP my_physical_node.my_domain.com.53367 >=20 >> my_master.35872: . 331777:333225(1448) ack 2613 win 6 E.....@.<... >> I.. >> B...w. ."..CC1L........... >> ...?..-M4LogicalNodeManager.my_physical_node.my_domain.com.name....my >> _physical_node.my_domain.com...> y_domain.com.physicalnode....my_physical_node.my_domain.com> _physical_node.my_domain.com>=20 >> .......> NullSource....type... >> ...lSource >> ............name....NullSink....type....NullSinksr0202_apps.NullSink >> ............name...sr0202.aus.optiver.com.my_logical_node_apps.NullSo >> urce >> NullSource....type... >> ...lSource >> ............name....NullSink....type....NullSinksr0202.aus.optiver.co >> m.NullSink ...........(pn-my_logical_node.aus.optiumber of=20 >> bytes........ >> 11:58:22.630891 IP my_physical_node.my_domain.com.53367 >=20 >> my_master.35872: P 333225:333700(475) ack 2613 win 6=20 >> =20 >> E.....@.<..c >> I.. >> ..w. .".:CC1L........... >> ............rt.starttime....Thu Jan 17 11:47:07 EST 2013... rt.vmnam= e...!Java HotSpot(TM) 64-Bit Server VM....name...(pn-my_physical_node.my_do= main.com.jvm-Info....rt.vmversion....16.3-b01....rt.vmvendor....Sun Microsy= stems Inc. >> ..........max.....@......mem.other.used................mem.other.committ= ed.....s......mem.heap.max........... >> >> ________________________________ >> Information contained in this communication (including any attachments) = is confidential and may be privileged or subject to copyright. If you have = received this communication in error you are not authorised to use the info= rmation in any way and Optiver requests that you notify the sender by retur= n email, destroy all copies and delete the information from your system. Op= tiver does not represent, warrant or guarantee that this communication is f= ree from computer viruses or other defects or that the integrity of this co= mmunication has been maintained. Any views expressed in this communication = are those of the individual sender. Optiver does not accept liability for a= ny loss or damage caused directly or indirectly by this communication or it= s use. >> >> Please consider the environment before printing this email. >> >> > -- > Alexander Alten-Lorenz > http://mapredit.blogspot.com > German Hadoop LinkedIn Group: http://goo.gl/N8pCF > > > -- Jeong-shik Jang / jsjang@gruter.com Gruter, Inc., R&D Team Leader www.gruter.com Enjoy Connecting