Return-Path: Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: (qmail 49831 invoked from network); 8 Dec 2010 18:56:25 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 8 Dec 2010 18:56:25 -0000 Received: (qmail 69227 invoked by uid 500); 8 Dec 2010 18:56:24 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 69146 invoked by uid 500); 8 Dec 2010 18:56:23 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 69137 invoked by uid 99); 8 Dec 2010 18:56:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 18:56:23 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [140.211.11.9] (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 08 Dec 2010 18:56:21 +0000 Received: (qmail 49756 invoked by uid 99); 8 Dec 2010 18:55:59 -0000 Received: from localhost.apache.org (HELO [192.168.168.16]) (127.0.0.1) (smtp-auth username cutting, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 18:55:59 +0000 Message-ID: <4CFFD4BE.9050704@apache.org> Date: Wed, 08 Dec 2010 10:55:58 -0800 From: Doug Cutting User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: general@hadoop.apache.org Subject: Re: [VOTE] Direction for Hadoop development References: <0E66662F-E044-4738-8C46-EA0EBCE674E0@yahoo-inc.com> <4CFE6D41.3050809@apache.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org On 12/07/2010 10:26 AM, Konstantin Shvachko wrote: >> I no longer think we should add any new serialization implementations to > the kernel. > > Not clear. Do you propose to keep current serialization(s) and not add new > ones? > Or do you propose to replace current serialization by abstract interfaces > and move implementations to libraries? We can't move existing serialization implementations to an optional library without breaking compatibility. Long-term that might be nice, but I am not proposing that short-term. Short-term I propose we avoid adding new serialization implementations to the default classpath, especially those that add new dependencies to every task. Long-term we might split library code into perhaps a few categories: - mandatory: this might include, e.g., IdentityMapper and IdentityReducer, the default implementations. - back-compatible: the collection of library components that were provided on the default classpath and can be enabled for back-compatible behavior. - optional: components that jobs can optionally depend on. This is where new components that are not mandatory would be added. Doug