hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anurag tangri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2001) Enhancement to SequenceFileOutputFormat to allow user to set MetaData
Date Wed, 08 Jun 2011 18:36:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046127#comment-13046127
] 

anurag tangri commented on MAPREDUCE-2001:
------------------------------------------

Hi,
Any update on this ? We are facing the same issue as David and I agree it is a common enough
request.

-Anurag

> Enhancement to SequenceFileOutputFormat to allow user to set MetaData
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2001
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2001
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.20.2
>            Reporter: David Rosenstrauch
>            Priority: Minor
>
> The org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat class currently does
not provide a way for the user to pass in a MetaData object to be written to the SequenceFile.
> Currently he only way for a developer to implement this functionality appears to be to
create a subclass which overrides the SequenceFileOutputFormat's getRecordWriter() method,
which is a bit of a kludge.
> This seems to be a common enough request to warrant a fix of some sort.  (It's already
been brought up twice in the past year:  http://www.mail-archive.com/common-user@hadoop.apache.org/msg02198.html
and http://www.mail-archive.com/mapreduce-user@hadoop.apache.org/msg00904.html)
> A couple of possible solutions:
> 1) provide a static method SequenceFileOutputFormat.setMetaData(Job, MetaData)
> 2) Provide a (non-static) setMetaData() method on the SequenceFileOutputFormat class.
 The user would create a subclass of SequenceFileOutputFormat which, say, implements Configurable.
 Then in the setConf() method, the user could create the MetaData object (using data from
the Configuration), and then call setMetaData.  The SequenceFileOutputFormat would then use
this MetaData object when creating the SequenceFile.  (Note that the user would have to create
a subclass of SequenceFileOutputFormat to make this solution work.)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message