Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Wed, 26 Mar 2014 09:03:15 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12703444.1395750902718.7370.1395824595898@arcas>
In-Reply-To: <JIRA.12703444.1395750902718@arcas>
References: <JIRA.12703444.1395750902718@arcas>
Subject: [jira] [Updated] (YARN-1871) We should eliminate writing *PBImpl
 code in YARN
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wangda Tan updated YARN-1871:
-----------------------------

    Attachment: YARN-1871.demo.patch

Attached a demo patch according to idea #3, (see YARN-1871.demo.patch)
bq. 3. Similar to @AtMostOnce annotation, make the ser-de as a runtime behavior.
The demo patch consists 3 parts,

*1. Changes on record class, (AllocateRequest in this patch)*
Make record class become a non-abstract class, add simple getters/setters implementation. 
And we need know that if is a variable set by user or not. So we consider for non-primitive fields, null is not set. For primitives, we consider MIN_VALUE is not set.

*2. Add a demo implementation (not completed) of serialization a record to Proto type using reflection*
See ProtoSerdeUtils.getProto

*3. Changes in *PBClientImpl using our ser-de utils*
See ApplicationMasterProtocolPBClientImpl.allocate

Looking forward your opinions on this!

> We should eliminate writing *PBImpl code in YARN
> ------------------------------------------------
>
>                 Key: YARN-1871
>                 URL: https://issues.apache.org/jira/browse/YARN-1871
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: api
>    Affects Versions: 2.4.0
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-1871.demo.patch
>
>
> Currently, We need write PBImpl classes one by one. After running "find . -name "*PBImpl*.java" | xargs wc -l" under hadoop source code directory, we can see, there're more than 25,000 LOC. I think we should improve this, which will be very helpful for YARN developers to make changes for YARN protocols.
> There're only some limited patterns in current *PBImpl,
> * Simple types, like string, int32, float.
> * List<?> types
> * Map<?> types
> * Enum types
> Code generation should be enough to generate such PBImpl classes.
> Some other requirements are,
> * Leave other related code alone, like service implemention (e.g. ContainerManagerImpl).
> * (If possible) Forward compatibility, developpers can write their own PBImpl or genereate them.


--
This message was sent by Atlassian JIRA
(v6.2#6252)