Return-Path: X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1AF5CD0FB for ; Wed, 17 Oct 2012 04:02:09 +0000 (UTC) Received: (qmail 33699 invoked by uid 500); 17 Oct 2012 04:02:08 -0000 Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org Received: (qmail 33477 invoked by uid 500); 17 Oct 2012 04:02:05 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-issues@hadoop.apache.org Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 33233 invoked by uid 99); 17 Oct 2012 04:02:04 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Oct 2012 04:02:04 +0000 Date: Wed, 17 Oct 2012 04:02:04 +0000 (UTC) From: "liu yu (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: <1281474853.55650.1350446524273.JavaMail.jiratomcat@arcas> Subject: [jira] [Commented] (MAPREDUCE-2001) Enhancement to SequenceFileOutputFormat to allow user to set MetaData MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477585#comment-13477585 ] liu yu commented on MAPREDUCE-2001: ----------------------------------- When you watch the problem, you need to check your reduce. It should be 'Iterable',not 'Iterator'. The latter won't rewrite the mothed reduce(...) of Reducer. > Enhancement to SequenceFileOutputFormat to allow user to set MetaData > --------------------------------------------------------------------- > > Key: MAPREDUCE-2001 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2001 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Affects Versions: 0.20.2 > Reporter: David Rosenstrauch > Priority: Minor > Attachments: MAPREDUCE-2001.patch > > > The org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat class currently does not provide a way for the user to pass in a MetaData object to be written to the SequenceFile. > Currently he only way for a developer to implement this functionality appears to be to create a subclass which overrides the SequenceFileOutputFormat's getRecordWriter() method, which is a bit of a kludge. > This seems to be a common enough request to warrant a fix of some sort. (It's already been brought up twice in the past year: http://www.mail-archive.com/common-user@hadoop.apache.org/msg02198.html and http://www.mail-archive.com/mapreduce-user@hadoop.apache.org/msg00904.html) > A couple of possible solutions: > 1) provide a static method SequenceFileOutputFormat.setMetaData(Job, MetaData) > 2) Provide a (non-static) setMetaData() method on the SequenceFileOutputFormat class. The user would create a subclass of SequenceFileOutputFormat which, say, implements Configurable. Then in the setConf() method, the user could create the MetaData object (using data from the Configuration), and then call setMetaData. The SequenceFileOutputFormat would then use this MetaData object when creating the SequenceFile. (Note that the user would have to create a subclass of SequenceFileOutputFormat to make this solution work.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira