hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-980) Modify JobHistory to use Avro for serialization instead of raw JSON
Date Thu, 17 Sep 2009 06:33:57 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756382#action_12756382
] 

Sharad Agarwal commented on MAPREDUCE-980:
------------------------------------------

It would have been ideal if we could directly use Avro generated classes without wrapping
them. Wrapper classes are not very maintainable because we need to modify at two places- schema
definition and wrapper class. Any reason why we can't directly use generated classes ? From
what I can think of  - this is done because we want all event classes to have a base interface,
constructor and field getters. Having constructor and getters should be straight forward in
code generator. For base class/interface, I think Avro can generate code from a template.
SpecificRecordBase methods can directly go into the generated class (to work around multiple
inheritance in java). Users can define the base class, interfaces or additional methods in
the template which can be used to generate Avro specific class. I understand that this may
not be doable at this point but something worth considering at some point to make Avro code
generation feature more compelling.



> Modify JobHistory to use Avro for serialization instead of raw JSON
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-980
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-980
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Jothi Padmanabhan
>            Assignee: Doug Cutting
>         Attachments: MAPREDUCE-980.patch, MAPREDUCE-980.patch, MAPREDUCE-980.patch
>
>
> MAPREDUCE-157 modifies JobHistory to log events using Json Format.  This can be modified
to use Avro instead. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message