Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 85513 invoked from network); 5 Nov 2008 14:00:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 5 Nov 2008 14:00:27 -0000 Received: (qmail 89916 invoked by uid 500); 5 Nov 2008 14:00:32 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 89874 invoked by uid 500); 5 Nov 2008 14:00:32 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 89863 invoked by uid 99); 5 Nov 2008 14:00:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Nov 2008 06:00:32 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.142.188] (HELO ti-out-0910.google.com) (209.85.142.188) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Nov 2008 13:59:12 +0000 Received: by ti-out-0910.google.com with SMTP id d27so1119tid.9 for ; Wed, 05 Nov 2008 05:59:43 -0800 (PST) Received: by 10.110.105.5 with SMTP id d5mr769508tic.37.1225893582810; Wed, 05 Nov 2008 05:59:42 -0800 (PST) Received: by 10.110.62.6 with HTTP; Wed, 5 Nov 2008 05:59:42 -0800 (PST) Message-ID: Date: Wed, 5 Nov 2008 22:59:42 +0900 From: "Edward J. Yoon" Sender: edward@udanax.org To: core-dev@hadoop.apache.org Subject: Re: [proposal] RAgzip: multiple map tasks for a large gzipped file In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 Content-Disposition: inline References: <11F139F761E3F945A1444102BB4CBFA79CBF80@MEXKR02.all.nhncorp.com> X-Google-Sender-Auth: 0eb7ea57520256c1 X-Virus-Checked: Checked by ClamAV on apache.org Pj4gIC0gc29sdmVzIHpsaWIgdmVyc2lvbiBjb25mbGljdCBwcm9ibGVtIGJ5IHN0YXRpYyBsaW5r aW5nIHpsaWIgMS4yLjMuCgpPaCwgT0suIEkgTWlzc2VkIEl0IQoKL0VkCgpPbiBXZWQsIE5vdiA1 LCAyMDA4IGF0IDEwOjQwIFBNLCBFZHdhcmQgSi4gWW9vbiA8ZWR3YXJkeW9vbkBhcGFjaGUub3Jn PiB3cm90ZToKPiBIaSwgd2VsY29tZSB5b3VyIGNvbnRyaWJ1dGUgOikKPgo+IEhlcmUncyBteSBm ZXcgY29tbWVudHMsCj4KPiAxKSBXZSBjYW4ndCBkaXN0cmlidXRlIGFueSBHUEwgb3IgTEdQTCBw cm9kdWN0cyB3aXRoIEhhZG9vcC4gQUZBSUssCj4gemxpYiB3YXMgdW5kZXIgYSBsaWNlbnNlIHRo YXQgcHVyZSBHUEwuIFNob3VsZCBpdCBiZSBuZWVkIGEgemxpYiBpbgo+IGxpYiBmb2xkZXI/Cj4g MikgWWVzLCB5b3UgY2FuIGNyZWF0ZSBKaXJhIGlzc3VlIGZvciB0aGlzIHRoaW5nLiBJZiB5b3Ug YXR0YWNoIHlvdXIKPiBwYXRjaCBhbmQgc3VibWl0IHBhdGNoLCBpdCdsbCBiZSByZXZpZXdlZCBi eSBhY3RpdmUgY29tbWl0dGVycy4KPgo+IFlvdXJzLAo+IEVkd2FyZAo+Cj4gMjAwOC8xMS81IOq5 gOuMgO2YhFvroZzqt7jrqqjrjbjrp4FdIDxkYWVoeXVuLmtpbUBuaG5jb3JwLmNvbT46Cj4+IEhl bGxvLAo+Pgo+PiBJJ20gbmV3IHRvIHRoaXMgbWFpbGluZyBsaXN0LCBhbmQgdGhpcyBpcyB0aGUg Zmlyc3QgdHJpYWwgb2YgY29udHJpYnV0aW9uLgo+Pgo+Pgo+Pgo+PiBXZSBoYXZlIG1hZGUgYSBw YXRjaCB0aGF0IGVuYWJsZXMgbXVsdGlwbGUgbWFwIHRhc2tzIGZvciBvbmUgbGFyZ2UgKmd6aXBw ZWQqIGZpbGUuIFdlIGNhbGwgdGhlIHBhdGNoIFJBZ3ppcCwgd2hpY2ggaXMgdGhlIGFiYnJldmlh dGlvbiBvZiBSYW5kb20gQWNjZXNzIGd6aXAuIEl0IGlzIGxpa2UgSEFET09QLTM2NDYsIHdoaWNo IHN1cHBvcnRzIGEgYmlnIGJ6aXAyIGZpbGUsIGFuZCBpcyBhbiBhbHRlcm5hdGl2ZSBhcHByb2Fj aCBvZiBQSUctNDIgd2hpY2ggcmVxdWlyZXMgcmUtY29tcHJlc3Npb24uCj4+Cj4+Cj4+Cj4+IFJB Z3ppcCB1c2VzIHpsaWIncyBpbmZsYXRlUHJpbWUgZnVuY3Rpb24gd2hpY2ggc3VwcG9ydHMgcmFu ZG9tIGFjY2VzcyBvbiBhIGd6aXBwZWQgZmlsZS4gU2luY2UgdGhlIGluZmxhdGVQcmltZSBpcyBz dXBwb3J0ZWQgZnJvbSB0aGUgdmVyc2lvbiBvZiAxLjIuMi40LCBpdCByZXF1aXJlcyB6bGliIDEu Mi4yLjQgb3IgaGlnaGVyLiAoV2UgdGVzdGVkIG9uIHpsaWIgMS4yLjMpCj4+Cj4+Cj4+Cj4+IFJB Z3ppcCByZXF1aXJlcyB0aGUgcHJlcHJvY2Vzc2luZyBzdGVwIHRoYXQgY3JlYXRlcyBhbiBhY2Nl c3MgcG9pbnQgKC5hcCkgZmlsZSwgd2hpY2ggaXMgbGlrZSB0aGUgaW5kZXggb2YgdGhlIGd6aXBw ZWQgZmlsZSBjaHVua3MuIChVbmZvcnR1bmF0ZWx5LCB0aGUgcHJlcHJvY2Vzc2luZyBzdGVwIHNl ZW1zIHRvIGJlIHNlcXVlbnRpYWwsIHRoYXQgaXMsIHdlIGNhbm5vdCBmaW5kIHRoZSB3YXkgdG8g cGFyYWxsZWxpemUuKQo+Pgo+Pgo+Pgo+PiBSQWd6aXAgc3BsaXRzIHRoZSBnemlwcGVkIGZpbGUg dXNpbmcgdGhlIC5hcCBmaWxlLiBUbyBiZSBtb3JlIHNwZWNpZmljLCBSQWd6aXAgcmVhZHMgdGhl IC5hcCBmaWxlLCBnZXQgdGhlIHN0YXJ0IHBvc2l0aW9uIGFuZCB0aGUgY29tcHJlc3Npb24gaW5m b3JtYXRpb24gb2YgYSBwYXJ0aXRpb24gb2YgdGhlIGd6aXBwZWQgZmlsZSwgZGVjb21wcmVzcyB0 aGUgcGFydGl0aW9uIGFuZCBmZWVkIGl0IHRvIHRoZSBtYXAgdGFzayBpbnB1dCB3aGVuIGEgbWFw IHRhc2sgc3RhcnRzLgo+Pgo+Pgo+Pgo+PiBJbiBzaG9ydCwgeW91IG1heSB1c2UgUkFnemlwIGJ5 IGp1c3QgY2hhbmdpbmcgSW5wdXRGb3JtYXQgdG8gUkFHWklQSW5wdXRGb3JtYXQuCj4+Cj4+Cj4+ Cj4+IFdlIGhhdmUgbWFkZSBSQWd6aXAgaW4gdHdvIHBhY2thZ2UgdHlwZXMgYXMgZm9sbG93czoK Pj4KPj4gMS4gamFyCj4+Cj4+IC0gZG9lcyBub3QgdG91Y2ggdGhlIEhhZG9vcCBjb3JlCj4+Cj4+ ICAtIHNvbHZlcyB6bGliIHZlcnNpb24gY29uZmxpY3QgcHJvYmxlbSBieSBzdGF0aWMgbGlua2lu ZyB6bGliIDEuMi4zLgo+Pgo+PiAyLiBoYWRvb3AgcGF0Y2gKPj4KPj4gLSBpbnRlZ3JhdGVkIGlu dG8gSGFkb29wIGNvcmUKPj4KPj4gLSBwYXRjaGVzIFpsaWJEZWNvbXByZXNzb3Iue2MsamF2YX06 IGxpYmhhZG9vcC5zbyBjaGFuZ2VzCj4+Cj4+ICAtIHRoZSB2ZXJzaW9uIG9mIHpsaWIgb24gdGhl IHN5c3RlbSBzaG91bGQgYmUgMS4yLjIuNCBvciBoaWdoZXIuCj4+Cj4+Cj4+Cj4+IFdoYXQgSSB3 YW50IHRvIGFzayBpczoKPj4KPj4gSG93IHRvIGNvbnRyaWJ1dGUgUkFnemlwIHRvIEhhZG9vcD8g TWF5IEkganVzdCBzdWJtaXQgdGhlIGhhZG9vcCBwYXRjaCAocGFja2FnZSAyKSB0byBKSVJBPwo+ Pgo+PiBJIGhhdmUgcmVhZCBodHRwOi8vd2lraS5hcGFjaGUub3JnL2hhZG9vcC9Ib3dUb0NvbnRy aWJ1dGUgYW5kIGNoYW5nZWQgb3VyIHNvdXJjZSBjb2RlIHRvIG1lZXQgdGhlIGNvZGluZyBzdHls ZS4KPj4KPj4KPj4KPj4gQW55IGNvbW1lbnRzIHdpbGwgYmUgYXBwcmVjaWF0ZWQuCj4+Cj4+IFRo YW5rIHlvdS4KPj4KPj4KPj4KPj4gLSBEYWVoeXVuIEtpbQo+Pgo+Pgo+Pgo+Pgo+Cj4KPgo+IC0t Cj4gQmVzdCBSZWdhcmRzLCBFZHdhcmQgSi4gWW9vbiBAIE5ITiwgY29ycC4KPiBlZHdhcmR5b29u QGFwYWNoZS5vcmcKPiBodHRwOi8vYmxvZy51ZGFuYXgub3JnCj4KCgoKLS0gCkJlc3QgUmVnYXJk cywgRWR3YXJkIEouIFlvb24gQCBOSE4sIGNvcnAuCmVkd2FyZHlvb25AYXBhY2hlLm9yZwpodHRw Oi8vYmxvZy51ZGFuYXgub3JnCg==