Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E0A0B9B91 for ; Sun, 25 Mar 2012 16:19:28 +0000 (UTC) Received: (qmail 62444 invoked by uid 500); 25 Mar 2012 16:19:27 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 62378 invoked by uid 500); 25 Mar 2012 16:19:27 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 62369 invoked by uid 99); 25 Mar 2012 16:19:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 25 Mar 2012 16:19:27 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of brock@cloudera.com designates 209.85.216.176 as permitted sender) Received: from [209.85.216.176] (HELO mail-qc0-f176.google.com) (209.85.216.176) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 25 Mar 2012 16:19:22 +0000 Received: by qcsd1 with SMTP id d1so3540012qcs.35 for ; Sun, 25 Mar 2012 09:19:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding:x-gm-message-state; bh=uW0ez9SCRNhmQL/53En9u+FdM6aTNEIrOIMBweXNmkU=; b=NG33WtRyYYve07WHn3rCbseFT3fkJUv2boX5rPOZGFlW2nxSXqmZdHKshXIoy4RghE SpjguiaLGPD4ZFEpzOQSs8og6SCIFSmQnHcK7TZTGOvBowF/4khOitLTWSHMmMLMCRcY I1hzDXqd7r+88YFrH3o+mSnGqGw+MR294qR5Kptg5rd20yjeNNgtzo54lS+00OsweP9m mAFwpWVFbgBKcdsu5JyB+UvGh7BmMsfOwZCS6t3KMHmeTQunUoKdL7cOivsohDfzCS7H 6YrbIws6/fpUxxv6lqoRQMSpZn5wm7fA7goF+5HHXLH1Biq5+FNrXqeFCGjfAqUxMhfH qriw== MIME-Version: 1.0 Received: by 10.224.179.193 with SMTP id br1mr24317515qab.56.1332692341433; Sun, 25 Mar 2012 09:19:01 -0700 (PDT) Received: by 10.229.140.21 with HTTP; Sun, 25 Mar 2012 09:19:01 -0700 (PDT) In-Reply-To: <4F6EF060.7010903@filez.com> References: <4F6EF060.7010903@filez.com> Date: Sun, 25 Mar 2012 09:19:01 -0700 Message-ID: Subject: Re: class usage counter in MapWritable From: Brock Noland To: mapreduce-user@hadoop.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQntvb6a2ihzyntjDrYPPWD3ptu8JvnQLRLuxJadgscy+DR8+fWJj0rQfSaFLil2U2xxCHIm X-Virus-Checked: Checked by ClamAV on apache.org Hi, inline... 2012/3/25 Radim Kolar : > current implementation of MapWritable and AbstractMapWritable do not trac= k > class usage. Class name is still serialized in write() to disk even if no > instance of such class exists in stored table anymore. > > I propose to add usage tracking to =A0AbstractMapWritable addToMap(Class > clazz) and provide function removeFromMap which will be called by > MapWritable. I can code such patch,. but i am inexperienced in hadoop > development, What hadoop branch should i target? I would first search to see if a JIRA (https://issues.apache.org/jira/secure/Dashboard.jspa) has already been raised and if not create one. Generally you want to target trunk with your patch. Much more information here: http://wiki.apache.org/hadoop/HowToContribute Cheers, Brock --=20 Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit= /