Return-Path: Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: (qmail 23219 invoked from network); 20 Jul 2010 02:02:28 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 20 Jul 2010 02:02:28 -0000 Received: (qmail 70069 invoked by uid 500); 20 Jul 2010 02:02:27 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 70007 invoked by uid 500); 20 Jul 2010 02:02:26 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 69999 invoked by uid 99); 20 Jul 2010 02:02:26 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Jul 2010 02:02:26 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of msegel@navteq.com designates 204.120.70.37 as permitted sender) Received: from [204.120.70.37] (HELO xmailfargo.navteq.com) (204.120.70.37) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Jul 2010 02:02:20 +0000 Received: from fargo-spamfw.navteq.com (navteq-spamfw.navteq.com [10.6.10.131]) by xmailfargo.navteq.com (8.13.6/8.13.6) with ESMTP id o6K1bk3v029735 for ; Mon, 19 Jul 2010 20:37:46 -0500 Received: from imailfargo.navteq.com (localhost [127.0.0.1]) by fargo-spamfw.navteq.com (Spam & Virus Firewall) with ESMTP id 6305647F6885 for ; Mon, 19 Jul 2010 20:58:49 -0500 (CDT) Received: from imailfargo.navteq.com (imailfargo.navteq.com [10.6.10.130]) by fargo-spamfw.navteq.com with ESMTP id FHAxFdiwDztaRfAy for ; Mon, 19 Jul 2010 20:58:49 -0500 (CDT) Received: from hq-ex-ht02.ad.navteq.com ([10.8.222.172]) by imailfargo.navteq.com (8.13.6/8.13.6) with ESMTP id o6K1wn5W000582 for ; Mon, 19 Jul 2010 20:58:49 -0500 Received: from hq-ex-mb02.ad.navteq.com ([fe80::880d:318e:e860:a8db]) by hq-ex-ht02.ad.navteq.com ([fe80::24f6:ca50:734d:df6e%14]) with mapi; Mon, 19 Jul 2010 20:58:49 -0500 From: "Segel, Mike" To: "general@hadoop.apache.org" Date: Mon, 19 Jul 2010 20:58:47 -0500 Subject: RE: Dynamic counters in Hadoop M/R jobs... Thread-Topic: Dynamic counters in Hadoop M/R jobs... Thread-Index: AcsnoDI6be7U+cKKSqeUfaaJFVRQ6QACJOZQ Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-NAIMIME-Disclaimer: 1 X-NAIMIME-Modified: 1 X-Virus-Checked: Checked by ClamAV on apache.org VGhhbmtzIGZvciB0aGUgcmVwbHkuDQoNCkdvdCBhIGNvdXBsZSBvZiBxdWVzdGlvbnMuLi4NCg0K SSBzYXcgdGhhdCBJIGNvdWxkIGdldCB0aGUgU3RhdHVzUmVwb3J0ZXIgZnJvbSB0aGUgTWFwcGVy LkNvbnRleHQuDQpCdXQgdGhlIEFQSSBkb2Vzbid0IGhhdmUgYSB3YXkgdG8gZ2V0IHRoZSBTdGF0 dXNSZXBvcnQgZnJvbSB0aGUgTWFwcGVyLkNvbnRleHQuDQoNCklmIHlvdSdyZSBnb2luZyB0byBn byB0byB0aGUgTWFwcGVyLkNvbnRleHQgeW91IGNhbiBnZXQgdGhlIGNvdW50ZXJzLg0KQW5kIHlv dSBoYXZlIGEgZ2V0Q291bnRlcihTdHJpbmcsIFN0cmluZykuIFRoZW4geW91IGNhbiBhbHdheXMg aW5jcmVtZW50IGl0Lg0KDQpCdXQgdGhpcyBraW5kIG9mIGJlZ3MgYSBxdWVzdGlvbiB3aGljaCBp c24ndCBkb2N1bWVudGVkLi4uDQoNCklmIEkgaGF2ZSBhIG1hcHBlci5Db250ZXh0IGFuZCBJIHNh eSBnZXRDb3VudGVyKCJGb28iLCJGb29iYXIgQ291bnQiKTsNCldoYXQgaGFwcGVucyBpZiB0aGVy ZSBpc24ndCBhIGNvdW50ZXIgKCJGb28iLCJGb29iYXIgQ291bnQiKSA/DQoNCldoYXQgSSBtZWFu IGlzIGhvdyBkbyB5b3UgYWRkIGEgZHluYW1pYyBjb3VudGVyIG9yIGlmIEkgYXNrIGZvciBhIGNv dW50ZXIgdGhhdCBkb2Vzbid0IGFscmVhZHkgZXhpc3QsIHdpbGwgaXQgY3JlYXRlIGl0Pw0KKFRo aXMgaXNuJ3QgZG9jdW1lbnRlZC4uLiBhbmQgaXRzIG5vdCBzYWZlIHRvIGFzc3VtZSBhbnl0aGlu Zy4pDQoNClNvcnJ5LCBidXQgZHluYW1pYyBjb3VudGVycyBpc24ndCByZWFsbHkgd2VsbCBkb2N1 bWVudGVkLg0KDQotTWlrZQ0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogYW50 aG9ueS51cnNvQGdtYWlsLmNvbSBbbWFpbHRvOmFudGhvbnkudXJzb0BnbWFpbC5jb21dIE9uIEJl aGFsZiBPZiBBbnRob255IFVyc28NClNlbnQ6IE1vbmRheSwgSnVseSAxOSwgMjAxMCA3OjExIFBN DQpUbzogZ2VuZXJhbEBoYWRvb3AuYXBhY2hlLm9yZw0KU3ViamVjdDogUmU6IER5bmFtaWMgY291 bnRlcnMgaW4gSGFkb29wIE0vUiBqb2JzLi4uDQoNClRoZSBuZXcgTWFwcGVyIGNsYXNzIHBhc3Nl cyBhIE1hcC5Db250ZXh0IG9iamVjdCB0byB0aGUgbWFwKCkgbWV0aG9kLg0KRnJvbSB0aGlzIHlv dSBjYW4gZ2V0IGEgU3RhdHVzUmVwb3J0ZXIgb2JqZWN0ICB3aGljaCAgY2FuIHByb2R1Y2UgYQ0K bmFtZWQgQ291bnRlciBvYmplY3QgYW5kIGluY3JlbWVudCBpdC4NCg0KaHR0cDovL2hhZG9vcC5h cGFjaGUub3JnL2NvbW1vbi9kb2NzL2N1cnJlbnQvYXBpL29yZy9hcGFjaGUvaGFkb29wL21hcHJl ZHVjZS9NYXBwZXIuQ29udGV4dC5odG1sDQpodHRwOi8vaGFkb29wLmFwYWNoZS5vcmcvY29tbW9u L2RvY3MvY3VycmVudC9hcGkvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkdWNlL1N0YXR1c1JlcG9y dGVyLmh0bWwNCmh0dHA6Ly9oYWRvb3AuYXBhY2hlLm9yZy9jb21tb24vZG9jcy9jdXJyZW50L2Fw aS9vcmcvYXBhY2hlL2hhZG9vcC9tYXByZWR1Y2UvQ291bnRlci5odG1sDQoNCkFsdGVybmF0aXZl bHksIHRoZSBvbGQgQVBJcyBhcmUgc3VwcG9zZWQgdG8gYmUgdGVtcG9yYXJpbHkNCmRlLWRlcHJl Y2F0ZWQgaW4gdGhlIG5leHQgc3RhYmxlIHJlbGVhc2UsIHNvIHlvdSBjYW4gdXNlIHRoZW0gYXMN CmRlc2NyaWJlZCBieSB0aGF0IGJvb2suDQoNCk9uIE1vbiwgSnVsIDE5LCAyMDEwIGF0IDExOjIy IEFNLCBNaWNoYWVsIFNlZ2VsDQo8bWljaGFlbF9zZWdlbEBob3RtYWlsLmNvbT4gd3JvdGU6DQo+ DQo+IEhpLA0KPg0KPiBJbiBsb29raW5nIGF0ICJUaGUgRGVmaW5pdGl2ZSBHdWlkZSIgoHBncyAy MTEtMjE4IGxvb2tpbmcgYXQgdGhlIGRvY3VtZW50YXRpb24gZm9yIGNvdW50ZXJzIGhhdmUgYSBx dWVzdGlvbiBhYm91dCBob3cgdG8gdXNlIHRoZQ0KPg0KPiBkeW5hbWljIChTdHJpbmcsIFN0cmlu ZykgbWV0aG9kcyB0byBjcmVhdGUvYWNjZXNzIGNvdW50ZXJzLg0KPg0KPiBMb29raW5nIGF0IHRo ZSBJZGVudGl0eVRhYmxlTWFwIGNsYXNzLCB0aGUgZm9sbG93aW5nIG1hcCgpIG1ldGhvZCBleGlz dHMoKToNCj4gcHVibGljIHZvaWQgbWFwKEltbXV0YWJsZUJ5dGVzV3JpdGFibGUga2V5LA0KPiCg IKAgoCCgIKAgoCCgIKBSb3dSZXN1bHQgdmFsdWUsDQo+IKAgoCCgIKAgoCCgIKAgoG9yZy5hcGFj aGUuaGFkb29wLm1hcHJlZC5PdXRwdXRDb2xsZWN0b3I8SW1tdXRhYmxlQnl0ZXNXcml0YWJsZSxS b3dSZXN1bHQ+IG91dHB1dCwNCj4goCCgIKAgoCCgIKAgoCCgb3JnLmFwYWNoZS5oYWRvb3AubWFw cmVkLlJlcG9ydGVyIHJlcG9ydGVyKQ0KPiCgIKAgoCCgIHRocm93cyBJT0V4Y2VwdGlvblRoaXMg bWV0aG9kIGlzIGRlcHJlY2F0ZWQgaW4gMjAuNQ0KPg0KPiBTbyBob3cgY2FuIHlvdSBnZXQgdGhl IFJlcG9ydGVyIHNvIHlvdSBjYW4gdXNlIHRoZSAncmVwb3J0ZXIuaW5jckNvdW50ZXIoU3RyaW5n IGdyb3VwLFN0cmluZyBjb3VudGVyLCBsb25nIGFtb3VudCknDQo+IG1ldGhvZC4gKFNlZSBwZyAy MTQgY29kZSBmcmFnbWVudCkNCj4NCj4gSSBtdXN0IGJlIG1pc3Npbmcgc29tZXRoaW5nLg0KPg0K PiBUaHgNCj4NCj4gLU1pa2UNCj4NCj4NCj4NCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gVGhlIE5ldyBCdXN5IGlz IG5vdCB0aGUgdG9vIGJ1c3kuIENvbWJpbmUgYWxsIHlvdXIgZS1tYWlsIGFjY291bnRzIHdpdGgg SG90bWFpbC4NCj4gaHR0cDovL3d3dy53aW5kb3dzbGl2ZS5jb20vY2FtcGFpZ24vdGhlbmV3YnVz eT90aWxlPW11bHRpYWNjb3VudCZvY2lkPVBJRDI4MzI2OjpUOldMTVRBR0w6T046V0w6ZW4tVVM6 V01fSE1QOjA0MjAxMF80DQoNCg0KVGhlIGluZm9ybWF0aW9uIGNvbnRhaW5lZCBpbiB0aGlzIGNv bW11bmljYXRpb24gbWF5IGJlIENPTkZJREVOVElBTCBhbmQgaXMgaW50ZW5kZWQgb25seSBmb3Ig dGhlIHVzZSBvZiB0aGUgcmVjaXBpZW50KHMpIG5hbWVkIGFib3ZlLiAgSWYgeW91IGFyZSBub3Qg dGhlIGludGVuZGVkIHJlY2lwaWVudCwgeW91IGFyZSBoZXJlYnkgbm90aWZpZWQgdGhhdCBhbnkg ZGlzc2VtaW5hdGlvbiwgZGlzdHJpYnV0aW9uLCBvciBjb3B5aW5nIG9mIHRoaXMgY29tbXVuaWNh dGlvbiwgb3IgYW55IG9mIGl0cyBjb250ZW50cywgaXMgc3RyaWN0bHkgcHJvaGliaXRlZC4gIElm IHlvdSBoYXZlIHJlY2VpdmVkIHRoaXMgY29tbXVuaWNhdGlvbiBpbiBlcnJvciwgcGxlYXNlIG5v dGlmeSB0aGUgc2VuZGVyIGFuZCBkZWxldGUvZGVzdHJveSB0aGUgb3JpZ2luYWwgbWVzc2FnZSBh bmQgYW55IGNvcHkgb2YgaXQgZnJvbSB5b3VyIGNvbXB1dGVyIG9yIHBhcGVyIGZpbGVzLg0K