From hadoop-dev-return-3107-apmail-lucene-hadoop-dev-archive=lucene.apache.org@lucene.apache.org Mon Aug 14 20:09:09 2006 Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 30846 invoked from network); 14 Aug 2006 20:09:09 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 14 Aug 2006 20:09:09 -0000 Received: (qmail 80388 invoked by uid 500); 14 Aug 2006 20:09:09 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 80220 invoked by uid 500); 14 Aug 2006 20:09:09 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 80211 invoked by uid 99); 14 Aug 2006 20:09:08 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Aug 2006 13:09:08 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [209.237.227.198] (HELO brutus.apache.org) (209.237.227.198) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Aug 2006 13:09:08 -0700 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id F18F241001A for ; Mon, 14 Aug 2006 20:06:15 +0000 (GMT) Message-ID: <9194881.1155585975986.JavaMail.jira@brutus> Date: Mon, 14 Aug 2006 13:06:15 -0700 (PDT) From: "Doug Cutting (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Commented: (HADOOP-403) close method in a Mapper should be provided with OutputCollector and a Reporter In-Reply-To: <14218394.1154383873895.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/HADOOP-403?page=comments#action_12427956 ] Doug Cutting commented on HADOOP-403: ------------------------------------- I don't think we should make all future io.Closeable implementations deal with a mapred.OutputCollector. I would prefer any such methods were added to a new interface, perhaps mapred.Output. One can easily workaround this problem by specifying an alternate MapRunnable. Is this issue really so common that this workaround, or Runping's, do not suffice? > close method in a Mapper should be provided with OutputCollector and a Reporter > ------------------------------------------------------------------------------- > > Key: HADOOP-403 > URL: http://issues.apache.org/jira/browse/HADOOP-403 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.5.0 > Environment: all > Reporter: Milind Bhandarkar > Assigned To: Milind Bhandarkar > Fix For: 0.6.0 > > > For mappers with side-effects, or mappers that work as aggregators (i.e. no output on individual key-value pairs, but an aggregate output at the end of all key-value pairs), output should be performed in the close method. For this purpose, we need to supply output collector and reporter to the close method of Mapper. This involves interface change, though. Thoughts ? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira