Mailing-List: contact dev-help@apex.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CABAipVZ9qnpYDd6KweZc5uXQEPOzjoAdUAi72Jf3y+9jHgpWXw@mail.gmail.com>
References: 
 <CABAipVYjHOuc2gg5Au8+otVdhyVXnG94ppfn7tL3YP40SFcejw@mail.gmail.com>
	<CABAipVatLS7-b=rmGS8oiJRRDk2YmmBB1bajm8h1TqOSM+1sPg@mail.gmail.com>
	<CABAipVZfZF6GaHwvm-wn2FiGMzycJYzj2Vq0U9j7qpa01iUpeA@mail.gmail.com>
	<CABAipVaHOZs7y5Fhx0+aiJ67WjHn2x5eCaROdMTQEiZm8=9ctw@mail.gmail.com>
	<CABAipVZedk49gNDPrY1dt5AyuRt39h-xnb6RBTj4zvJyDSBdSA@mail.gmail.com>
	<CABAipVbLhiOnYFwh9bBdBuaFRh59TBL3DoHi=2kwNbXJJPQmoA@mail.gmail.com>
	<CABAipVbK6t9TK4epyz1b3m5s91KPnb3wEXVJ2ByuoRAK8+dzXw@mail.gmail.com>
	<CABAipVY+=15z7v00NxJ78A0Rhh=9_i-xmwv1R9YVu58wVJPeCQ@mail.gmail.com>
	<CABAipVZrDkoMWGj_B-Gq4VFUrYrUM=m=gQQrSSkNw=6eg4Pteg@mail.gmail.com>
	<CABAipVYqWUWMFMFo3euSACSSauWU4OOMcRjR2OMDGP2zrFKDzA@mail.gmail.com>
	<CAKJfLDN0tgfS8KqWYW9kfXNk7eEEs9Etp4GkvQn-Gu6c21hn3w@mail.gmail.com>
	<CABAipVZ9qnpYDd6KweZc5uXQEPOzjoAdUAi72Jf3y+9jHgpWXw@mail.gmail.com>
Date: Tue, 19 Apr 2016 22:57:21 -0700
Message-ID: 
 <CAKJfLDOku+MFgU46HT8GHF_k_Bz+vBQRfsUSV9sLouho2KMVBA@mail.gmail.com>
Subject: Re: AppFactory in Apex
From: Thomas Weise <thomas@datatorrent.com>
To: dev@apex.incubator.apache.org
Content-Type: text/plain; charset=UTF-8

#2 JSON format is already supported.

#4 is an example how you can translate a specification into a logical
plan without any change in core.

I think it is important to discuss this in the context of a use case.

Thomas

On Mon, Apr 18, 2016 at 10:42 PM, Chinmay Kolhatkar <chinmay@apache.org> wrote:
> Hi Thomas,
>
> Can you please elaborate on "different DAG specification can be translated
> into a Logical Plan using existing API"?
>
> I don't have specific usecase in mind. This is just an idea that I wanted
> to share with the community on following thought process:
>
> 1. With current AppFactory impl (for JSON, properties file and
> populateDAG), gives us those 3 ways to construct and configure DAG. Lets
> say one wants to have own DAG specification format, then that's not
> possible as of now.
>
> 2. JSON format opens a new way of creating DAG. To me, this should a great
> power of apex engine that DAG specification and configuration is not
> limited to Java code, but even more. Making a configurable AppFactory can
> open discussion to even better formats/standards for specifications of DAG.
>
> 3. If JSON/Properties file format needs a change either in DAG
> specification/configuration, this means, the changes has to go in Apex
> core. Instead, if we allow overriding AppFactory, one could have more DAG
> specifications/conf added to Malhar making lesser changes in apex
> core/engine. I'm thinking more in perspective of stability of apex engine
> release build.
>
> 4. Recently Siyuan shared high level APIs for apex application. Having
> a separate AppFactory for such high level APIs can make this more modular.
>
> 5. Another example where this could be useful is Apex DataFrames shared by
> Capital One folks.
>
> Overall, this is just an idea and is not bound by any specific usecase.
> The basic idea is
> - Plug-ability to apex engine
> - Make apex platform more extensible
> - Reducing configuration/DAG specification related changes making
> apex engine more sturdy.
>
>
> Thanks,
> Chinmay.
>
>
>
> On Tue, Apr 19, 2016 at 7:56 AM, Thomas Weise <thomas@datatorrent.com>
> wrote:
>
>> Chinmay,
>>
>> Different DAG specifications can be translated into a logical plan
>> using the existing API.
>>
>> AppFactory is an implementation detail. Can you describe the problem
>> you are looking to solve?
>>
>> Thanks
>>
>>
>> On Mon, Apr 18, 2016 at 6:21 PM, Chinmay Kolhatkar <chinmay@apache.org>
>> wrote:
>> > Dear Community,
>> >
>> > Does apex engine allows user to override or extend existing
>> implementation
>> > of AppFactory from outside of engine?
>> >
>> > If it not, would allowing it make any required launch time enhancements
>> > easier and safer (for stability) rather than changing apex engine code?
>> >
>> > Another advantage I see by doing this is, user can have their own DAG
>> > specification format if required.
>> >
>> > Thanks,
>> > Chinmay.
>>