hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: serializing/deserializing wrapped protobuf generated classes
Date Tue, 21 May 2013 15:32:26 GMT
I agree with you, and I started down the path of ripping out that
abstraction layer because, at this point, we are only going to support pro
to buffers, but it turned out to be too much work to get done before the
we wanted to lock down the APIs so I stopped.  I would suggest that you do
the minimum to keep the abstraction in place, but don't do a lot of work
beyond that.  Perhaps something similar to the RecordFactory API that
would allow you to serialize/deserialize a blob into an instance of a
known class.  Perhaps you could even extend RecordFactory to look
something like

public interface RecordFactory {
  public <T> T newRecordInstance(Class<T> clazz) throws YarnException;

public <T> T newRecordInstance(Class<T> clazz, byte[] data) throws

public <T> byte[] fromRecordInstance(T t) throws YarnException;


On 5/21/13 8:54 AM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:

>(phone typing)
>On May 20, 2013, at 11:04, Alejandro Abdelnur <tucu@cloudera.com> wrote:
>> [I know it has been discussed before the need (or not) of having the
>>current wrappers hiding the protobuf generated classes].
>> I order to do things like AMs failover and checkpointing I need to
>>serialize app IDs, app attempt IDs, containers and/or IDs,  resource
>>requests, etc.
>> This means that the current wrapping hides the PB impl, thus hiding the
>>provided ser/deser capabilities.
>> I could force-cast a record to ProtoBase (which is private) and then
>>get the PROTO Message and then do the ser/deser with that.
>> But this, IMO, is a no no.
>> Thoughts?
>> Thanks
>> -- 
>> Alejandro

View raw message