hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-1871) We should eliminate writing *PBImpl code in YARN
Date Wed, 26 Mar 2014 09:03:15 GMT

     [ https://issues.apache.org/jira/browse/YARN-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Wangda Tan updated YARN-1871:

    Attachment: YARN-1871.demo.patch

Attached a demo patch according to idea #3, (see YARN-1871.demo.patch)
bq. 3. Similar to @AtMostOnce annotation, make the ser-de as a runtime behavior.
The demo patch consists 3 parts,

*1. Changes on record class, (AllocateRequest in this patch)*
Make record class become a non-abstract class, add simple getters/setters implementation.

And we need know that if is a variable set by user or not. So we consider for non-primitive
fields, null is not set. For primitives, we consider MIN_VALUE is not set.

*2. Add a demo implementation (not completed) of serialization a record to Proto type using
See ProtoSerdeUtils.getProto

*3. Changes in *PBClientImpl using our ser-de utils*
See ApplicationMasterProtocolPBClientImpl.allocate

Looking forward your opinions on this!

> We should eliminate writing *PBImpl code in YARN
> ------------------------------------------------
>                 Key: YARN-1871
>                 URL: https://issues.apache.org/jira/browse/YARN-1871
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: api
>    Affects Versions: 2.4.0
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-1871.demo.patch
> Currently, We need write PBImpl classes one by one. After running "find . -name "*PBImpl*.java"
| xargs wc -l" under hadoop source code directory, we can see, there're more than 25,000 LOC.
I think we should improve this, which will be very helpful for YARN developers to make changes
for YARN protocols.
> There're only some limited patterns in current *PBImpl,
> * Simple types, like string, int32, float.
> * List<?> types
> * Map<?> types
> * Enum types
> Code generation should be enough to generate such PBImpl classes.
> Some other requirements are,
> * Leave other related code alone, like service implemention (e.g. ContainerManagerImpl).
> * (If possible) Forward compatibility, developpers can write their own PBImpl or genereate

This message was sent by Atlassian JIRA

View raw message