hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: serializing/deserializing wrapped protobuf generated classes
Date Tue, 21 May 2013 19:28:08 GMT
Sid,

I've opened YARN-710 for this, and posted a patch following Bobby's
suggestion.

Yes, this ought to be part of the public API.

Thanks.



On Tue, May 21, 2013 at 12:48 AM, Siddharth Seth
<seth.siddharth@gmail.com>wrote:

> I'd agree. Using ProtoBase in application code is not a good idea.
>
> The MR AM currently uses the string form of the IDs to write to the history
> file, and constructs them back from this. This would likely not be
> practical for some of the more complicated records like Container.
>
> I initially thought a serialization/deserialization interface on all
> records would be useful (especially in the context of RM state store), but
> have since been convinced otherwise - YARN does not need to expose it's
> underlying record serialization mechanism to user code - not via the
> individual records itself.  That said, should YARN be providing a utility
> library to help app developers with this functionality. This would end up
> becoming part of the public API, along with the data format. What do others
> think ?
>
> Thanks
> - Sid
>
>
>
> On Mon, May 20, 2013 at 11:04 AM, Alejandro Abdelnur <tucu@cloudera.com
> >wrote:
>
> > [I know it has been discussed before the need (or not) of having the
> > current wrappers hiding the protobuf generated classes].
> >
> > I order to do things like AMs failover and checkpointing I need to
> > serialize app IDs, app attempt IDs, containers and/or IDs,  resource
> > requests, etc.
> >
> > This means that the current wrapping hides the PB impl, thus hiding the
> > provided ser/deser capabilities.
> >
> > I could force-cast a record to ProtoBase (which is private) and then get
> > the PROTO Message and then do the ser/deser with that.
> >
> > But this, IMO, is a no no.
> >
> > Thoughts?
> >
> > Thanks
> >
> > --
> > Alejandro
> >
>



-- 
Alejandro

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message