Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 195E8E298 for ; Tue, 27 Nov 2012 19:50:38 +0000 (UTC) Received: (qmail 66174 invoked by uid 500); 27 Nov 2012 19:50:33 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 66012 invoked by uid 500); 27 Nov 2012 19:50:32 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 66005 invoked by uid 99); 27 Nov 2012 19:50:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2012 19:50:32 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [206.47.135.205] (HELO Spam1.prd.mpac.ca) (206.47.135.205) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2012 19:50:27 +0000 Received: from Spam1.prd.mpac.ca (unknown [127.0.0.1]) by IMSVA80 (Postfix) with ESMTP id E43331D805E for ; Tue, 27 Nov 2012 14:50:05 -0500 (EST) Received: from SMAIL1.prd.mpac.ca (unknown [172.29.2.53]) by Spam1.prd.mpac.ca (Postfix) with ESMTP id 9F2861D805D for ; Tue, 27 Nov 2012 14:50:05 -0500 (EST) Received: from SMAIL1.prd.mpac.ca ([fe80::d548:4221:967c:4cfb]) by SMAIL1.prd.mpac.ca ([fe80::18cb:8648:b77f:2b55%11]) with mapi id 14.02.0318.004; Tue, 27 Nov 2012 14:50:05 -0500 From: "Kartashov, Andy" To: "user@hadoop.apache.org" Subject: RE: block-size vs split-size Thread-Topic: block-size vs split-size Thread-Index: Ac3MsKx5nA4MvoXMQfGBmG6Ib3t9+QAQcfiAAAas0vA= Date: Tue, 27 Nov 2012 19:50:04 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.29.60.102] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-TM-AS-Product-Ver: IMSVA-8.0.0.1304-6.5.0.1024-19398.001 X-TM-AS-Result: No--25.809-5.0-31-10 X-imss-scan-details: No--25.809-5.0-31-10 X-TM-AS-Result-Xfilter: Match text exemption rules:No X-TMASE-MatchedRID: WMT2WRIkHPOnykMun0J1wrar4tnNxmAKh+w9Wz/xXDphb6wYa04uHXAW uORm0TIau958H+JYQswXIJWO/t2WjkndsLjTlFHfGPjYszIlSDyMs7LXcKBvj9KBqnreoHKhM+p +NpswZmpNI04Zf7ZHpou5LnX5kwATKeLGzKm/w9LkKCFOKwAEzMMdI0UcXEHz9ighU2asJv0cAJ RwO9xbIBn8e2zDL5jI7JW9IYR99sM1gIrQKBEHY0KcYi5Qw/RVT5++FIORChBV84HrPxCfbDGnN +rWrtPZ2NBcWhOm2bXtjJAYcEf6At2GnHnvtSec+lWD63Q3aXpwylaWfhCWnKtGJmiL6Hmb9tCR GNQsNpGPXlv0Nc1Idxun0Fro04lN7fcsGVlK1PSKQCqq8vYqFbN6CI3aOme/IYP4Wne9kdSW9wR pDRMjcnHHvM3MIz1JHdvJ//UlR8HjrkVJ/QftkzBCv+rYOiNxlfGwpNSPVn7UHQeTVDUrIgCIPW dCItZB X-Virus-Checked: Checked by ClamAV on apache.org VGhhbmtzIEhhcnNoLiBJIHRvdGFsbHkgZm9yZ290IGFib3V0IHRoZSBsb2NhbGl0eSB0aGluZy4N Cg0KSSB0YWtlIGl0LCBmb3IgdGhlIGJlc3QgcGVyZm9tYW5jZSBpdCBpcyBiZXR0ZXIgbGVhdmUg dGhlIHNwbGl0IHNpemUgcHJvcGVydHkgYWxvbmUgYW5kIGxldCB0aGUgZnJhbWV3b3JrIGhhbmRs ZSB0aGUgc3BsaXRzIG9uIHRoZSBiYXNpcyBvZiB0aGUgYmxvY2sgc2l6ZS4NCg0KcC5zLiBUaGVy ZSB3ZXJlIG1lYW50IHRvIGJlIG9ubHkgNSBxdWVzdGlvbnMuDQoNClJnZHMsDQpBSzQ3DQoNCg0K LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IEhhcnNoIEogW21haWx0bzpoYXJzaEBj bG91ZGVyYS5jb21dDQpTZW50OiBUdWVzZGF5LCBOb3ZlbWJlciAyNywgMjAxMiAxMjo1NyBQTQ0K VG86IDx1c2VyQGhhZG9vcC5hcGFjaGUub3JnPg0KU3ViamVjdDogUmU6IGJsb2NrLXNpemUgdnMg c3BsaXQtc2l6ZQ0KDQpIaSwNCg0KUmVzcG9uc2UgaW5saW5lLg0KDQpPbiBUdWUsIE5vdiAyNywg MjAxMiBhdCA4OjM1IFBNLCBLYXJ0YXNob3YsIEFuZHkgPEFuZHkuS2FydGFzaG92QG1wYWMuY2E+ IHdyb3RlOg0KPiBHdXlzLA0KPg0KPiBJIHVuZGVyc3RhbmQgdGhhdCBpZiBub3Qgc3BlY2lmaWVk LCBkZWZhdWx0IGJsb2NrIHNpemUgb2YgSERGcyBpcw0KPiA2NE1iLiBZb3UgY2FuIGNvbnRyb2wg dGhpcyB2YWx1ZSBieSBhbHRlcmluZyBkZnMuYmxvY2suc2l6ZSBwcm9wZXJ0eQ0KPiBhbmQgaW5j cmVhc2luZyB0byB2YWx1ZSB0byA2NE1iIHggMiBvciA2NE1iIHggNC4uIEV2ZXJ5IHRpbWUgd2Ug bWFrZQ0KPiB0aGUgY2hhbmdlIHRvIHRoaXMgcHJvcGVydHkgd2UgbXVzdCByZWltcG9ydCB0aGUg ZGF0YSBmb3IgdGhlIGNoYW5nZXMNCj4gdG8gdGFrZSBlZmZlY3QNCg0KWWVzLCB0byBjaGFuZ2Ug dGhlIERGUyBsZXZlbCBmaWxlIGJsb2NrIHNpemUsIHRoZSBmaWxlIGhhcyB0byBiZSByZS13cml0 dGVuIHdpdGggdGhlIG5ldyB2YWx1ZS4NCg0KPiBNeSBxdWVzdGlvbiBpcyBhYm91dCBzcGxpdCBz aXplLiBJIHVuZGVyc3RhbmQgaXQgaXMgdXNlZCBieSBNYXBSZWR1Y2UNCj4gdG8gYXNzaWduIHRh c2sgdG8gdGFza3RyYWNrZXJzOg0KPg0KPiAxLiAgICAgICBJcyBzcGxpdCBzaXplIChpZiBub3Qg c3BlY2lmaWVkIGluIHRoZSBwcm9wZXJ0eQ0KPiBtYXByZWQubWluLnNwbGl0LnNpemUpIGJ5IGRl ZmF1bHQgZXF1YWxzIHRvIHRoZSAgZGVmYXVsdCBibG9jay1zaXplIG9yIDY0TWI/DQoNCklmIHVu c3BlY2lmaWVkLCB0aGUgTVIgZnJhbWV3b3JrIGRvZXMgcmVseSBvbiB0aGUgcmV0cmlldmVkIGJs b2NrIHNpemVzIG9mIHRoZSBmaWxlIHRvIGNyZWF0ZSB0aGUgdGFza3MuIFNvIHllcywgaXQgZGVm YXVsdHMgdG8gdGhlIGZpbGUncyBibG9jayBzaXplIGlmIHlvdSB1c2UgSERGUy4NCg0KPiAyLiAg ICAgICBJZiB5b3UgaW5jcmVhc2VkIGJsb2NrIHNpemUgc2F5IHRvIDEyOE1iPyBXaWxsIHNwbGl0 IHNpemUgKGlmIG5vdA0KPiBzcGVjaWZpZWQpIGJlIGVxdWFsIHRvIDEyOE1iIG9mIGJsb2NrcyBz aXplIG9yIHdpbGwgaXQgcmVtYWluIGF0IDY0TWINCj4gb2YgZGVmYXVsdCBibG9jayBzaXplPw0K DQpTaW5jZSAoMSkgaXMgdHJ1ZSBhcyBhbiB1bnRvdWNoZWQgZGVmYXVsdCwgdGhlIHNwbGl0IHNp emUgd2lsbCBhbHNvIGNoYW5nZSB0byAxMjggTUIuDQoNCj4gMy4gICAgICAgSWYsIHNheSwgeW91 ciBpbnB1dCBmaWxlIGlzIDEyOE1iLiBBdCBkZWZhdWx0IDY0TWIgYmxvY2stc2l6ZSB5b3UNCj4g d2lsbCBnZXQgdGhlIGZpbGUgdG90YWxsaW5nIHR3byBibG9ja3MuIFlvdXIgSm9iVHJhY2tlciB3 aWxsIGNyZWF0ZQ0KPiB0d28gbWFwIG1hc2tzIOKAkyBvbmUgcGVyIGJsb2NrKeKApiBidXQgd2hh dCBpZiB5b3Ugc3BlY2lmaWVkDQo+IG1hcHJlZC5taW4uc3BsaXQuc2l6ZSBhdCAxMjhNYj8gSSBz dXBwb3NlLCBkZXNwaXRlIHRoZSBmYWN0IHRoZXJlIGFyZQ0KPiAyIGJsb2NrcyBpbiB0aGUgSERG UyB0aGVyZSB3aWxsIGJlIG9ubHkgb25lIGlucHV0IHNwbGl0IGZvciB0aGUNCj4gTWFwcmVkdWNl LiBJcyBtYXByZWQubWluLnNwbGl0LnNpemUgZGVzaWduZWQgdG8gb3ZlcnJpZGUgdGhlIGJsb2Nr LXNpemUgcHJvcGVydHkgd2hlbiBwcmVwYXJpbmcgaW5wdXRTcGxpdHM/DQoNCkNvcnJlY3QuDQoN CjEyOCBNQiBmaWxlLCB3aXRoIDY0IE1CIGJsb2NrIHNpemUgLS0+IERlZmF1bHRzIC0tPiAyIHRh c2tzDQoxMjggTUIgZmlsZSwgd2l0aCA2NCBNQiBibG9jayBzaXplIC0tPiBNaW4gc3BsaXQgc2l6 ZSAxMjggTUIgLS0+IDEgdGFzaw0KDQpZZXMgdGhlIHByb3BlcnR5IGlzIGRlc2lnbmVkIHRvIG92 ZXJyaWRlIHRoZSBzaW1wbGUgZGVmYXVsdCBvZiBibG9jayBzaXplIC0+IGlucHV0IHNwbGl0IHNp emUgbWFwcGluZy4NCg0KPiA0LiAgICAgICBHaXZlbiB0aGUgYWJpbGl0eSB0byBzZXQgIGJsb2Nr LXNpemUgYW5kIHNwbGl0IOKAkyBzaXplIGluZGl2aWR1YWxseSwNCj4gd2FzIHRoZSBtYWluIHB1 cnBvc2Ugb2YgdGhpcyB0byBnYWluIGJldHRlciBjb250cm9sIG9mIG9uZSBwcm9wZXJ0eQ0KPiBv dmVyIGFub3RoZXI/PyAgSWYgbXkgdW5kZXJzdGFuZGluZyBvbiB0aGUgM3JkIHBvaW50IGNvcnJl Y3QuIFRoZW4sDQo+IHNheSwgeW91IGltcG9ydGVkICB5b3VyIGRhdGEgYXQgMTI4TWIgYmxvY2st c2l6ZSBidXQgbGF0ZXIgcmVhbGlzZWQNCj4geW91IHNob3VsZCBoYXZlIGdvbmUgaGlnaGVy4oCm IGluc3RlYWQgb2YgcmUtaW1wb3J0aW5nIGFsbCB5b3VyIGRhdGEgYXQNCj4gMjU2TWIgcGVyIGJs b2NrLCB5b3UgY2FuIGNoYW5nZSBzcGxpdCBzaXplIHByb3BlcnR5IHRvIDI1Nk1iLiBBbSBJDQo+ IGdyYXNwaW5nIHRoaXMgY29uY2VwdCBjb3JyZWN0bHk/DQoNCldoaWxlIHlvdSBnZXQgdGhlIGNv bmNlcHQgb2Ygc3BsaXRzIHJpZ2h0LCB0aGVyZSdzIG9uZSBwb2ludCBhYm91dCBsb2NhbGl0eSB0 byBjb25zaWRlcjoNCg0KWW91IGNvdWxkIGRvIHRoYXQsIGJ1dCB5b3Ugd291bGQgbG9zZSBsb2Nh bGl0eS4gVGhlIHJlYXNvbiB0aGUgZGVmYXVsdCBzcGxpdCBhbGdvcml0aG0gc3RpY2tzIHRvIGJs b2NrIGJvdW5kYXJpZXMgaXMgc3VjaCB0aGF0IGVhY2ggdGFzayBpbmRpdmlkdWFsbHkgcHJvY2Vz c2VzIG9uZSBibG9jayBhbG9uZSwgYW5kIHRoZSBzY2hlZHVsZXIgY2FuIGRvIGEgbW9yZSBlZmZl Y3RpdmUgam9iIGluIG1ha2luZyB0aGUgdGFzayBydW4gd2hlcmUgdGhpcyBpbmRpdmlkdWFsIGJs b2NrIHJlc2lkZXMuDQoNCldoZW4geW91IG92ZXJyaWRlIG1pbi1zcGl0LXNpemUgYW5kIG1ha2Ug dGhlIHNwbGl0IGNhcnJ5IHR3byBibG9ja3Mgd29ydGggb2Ygb2Zmc2V0ICsgbGVuZ3RoLCB0aGVu IHRoZSB0d28gYmxvY2tzIGNvdWxkIGJlIHJlc2lkaW5nIGF0IGRpZmZlcmVudCBub2RlcyBidXQg dGhlIHRhc2sgd2lsbCBydW4gb25seSBhdCBvbmUgbm9kZSwgbGVhZGluZyB0byBub24tZGF0YS1s b2NhbCBwcm9jZXNzaW5nLCB3aGljaCBjb3VsZCBlbmQgdXAgYmVpbmcgc2xvd2VyLg0KDQo+IDUu DQoNCllvdSBtaXNzZWQgeW91ciA1dGggcXVlc3Rpb24/DQoNCi0tDQpIYXJzaCBKDQpOT1RJQ0U6 IFRoaXMgZS1tYWlsIG1lc3NhZ2UgYW5kIGFueSBhdHRhY2htZW50cyBhcmUgY29uZmlkZW50aWFs LCBzdWJqZWN0IHRvIGNvcHlyaWdodCBhbmQgbWF5IGJlIHByaXZpbGVnZWQuIEFueSB1bmF1dGhv cml6ZWQgdXNlLCBjb3B5aW5nIG9yIGRpc2Nsb3N1cmUgaXMgcHJvaGliaXRlZC4gSWYgeW91IGFy ZSBub3QgdGhlIGludGVuZGVkIHJlY2lwaWVudCwgcGxlYXNlIGRlbGV0ZSBhbmQgY29udGFjdCB0 aGUgc2VuZGVyIGltbWVkaWF0ZWx5LiBQbGVhc2UgY29uc2lkZXIgdGhlIGVudmlyb25tZW50IGJl Zm9yZSBwcmludGluZyB0aGlzIGUtbWFpbC4gQVZJUyA6IGxlIHByw6lzZW50IGNvdXJyaWVsIGV0 IHRvdXRlIHBpw6hjZSBqb2ludGUgcXVpIGwnYWNjb21wYWduZSBzb250IGNvbmZpZGVudGllbHMs IHByb3TDqWfDqXMgcGFyIGxlIGRyb2l0IGQnYXV0ZXVyIGV0IHBldXZlbnQgw6p0cmUgY291dmVy dHMgcGFyIGxlIHNlY3JldCBwcm9mZXNzaW9ubmVsLiBUb3V0ZSB1dGlsaXNhdGlvbiwgY29waWUg b3UgZGl2dWxnYXRpb24gbm9uIGF1dG9yaXPDqWUgZXN0IGludGVyZGl0ZS4gU2kgdm91cyBuJ8Oq dGVzIHBhcyBsZSBkZXN0aW5hdGFpcmUgcHLDqXZ1IGRlIGNlIGNvdXJyaWVsLCBzdXBwcmltZXot bGUgZXQgY29udGFjdGV6IGltbcOpZGlhdGVtZW50IGwnZXhww6lkaXRldXIuIFZldWlsbGV6IHBl bnNlciDDoCBsJ2Vudmlyb25uZW1lbnQgYXZhbnQgZCdpbXByaW1lciBsZSBwcsOpc2VudCBjb3Vy cmllbA0K