Mailing-List: contact dev-help@apex.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.incubator.apache.org
Subject: Re: Modules support in Apex
To: dev@apex.incubator.apache.org
References: 
 <CAD0KA=PGJYA-ztwEzkRh5y6soQ8YQTveuS6CjgdSSCH-wcXWfg@mail.gmail.com>
 <CAOeZVieNYFn4eW-1JJ4mTBtHS0WH+G7NdAYfzko2nRLVVLy-Mw@mail.gmail.com>
 <CAD0KA=PXEKfMa9ArD_XN_eSQM_DLzHXXourFFB2gAK+KV_t7+w@mail.gmail.com>
 <CAOeZVicypLW-vL+j4KR5ztXQXg+mTDnxXfz45tH+zAUXOmEvKQ@mail.gmail.com>
From: Vlad Rozov <v.rozov@datatorrent.com>
Organization: DataTorrent
Message-ID: <55E4C463.6020504@datatorrent.com>
Date: Mon, 31 Aug 2015 14:17:23 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0)
 Gecko/20100101 Thunderbird/38.2.0
MIME-Version: 1.0
In-Reply-To: 
 <CAOeZVicypLW-vL+j4KR5ztXQXg+mTDnxXfz45tH+zAUXOmEvKQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Atri,

As a first cut module is a predefined subgraph that can be inserted into 
a DAG. Generally speaking module is not required to expand to the same 
subgraph. Depending on module properties or configuration, module may 
expand to different subgraphs. At the same time, similar to an operator 
module has predefined input/output ports and properties.

Thank you,

Vlad

On 8/31/15 11:46, Atri Sharma wrote:
> No, I dont think APEX-3's functionality needs to be exceeded for this. What
> I am trying to understand here is the concept of Module. Is it a family of
> operators defined by same interface implementation *or* is it a defined
> subgraph that can be replaced and used as a part of a subgraph instead of
> building the whole connection again?
>
> On Mon, Aug 31, 2015 at 11:27 PM, Tushar Gosavi <tushar@datatorrent.com>
> wrote:
>
>> Yes, you are correct. The APEX-3 is for dynamic expansion of the DAG, If we
>> can
>> expand the DAG dynamically then we can use that functionality to expand the
>> DAG
>> with know list of operator and connections between them. In a way APEX-3
>> provides
>> bigger functionality than APEX-55.
>>
>> Let me know if you think that module functionality requires more support
>> that what will be provided by APEX-3.
>>
>>
>>
>> On Mon, Aug 31, 2015 at 10:39 AM, Atri Sharma <atri@apache.org> wrote:
>>
>>> Looks interesting.
>>>
>>> Can you explain a bit on how it helps APEX-3? The objective of APEX-3 is
>> to
>>> have dynamic expansion of sub DAGs. If I understand correctly, the static
>>> list is to be known when declaring a module.
>>>
>>> Please correct me if I am wrong.
>>>
>>> On Mon, Aug 31, 2015 at 11:05 PM, Tushar Gosavi <tushar@datatorrent.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> We are working on adding support for Modules in Apex. A module is a
>> group
>>>> of operators that will have their own existence and will ease the way
>> we
>>>> are currently defining an application.
>>>>
>>>> A module is defined by:
>>>>
>>>>     - list of operators.
>>>>     - list of input and output ports.
>>>>     - set of properties for the module.
>>>>     - set of attributes for the module.
>>>>
>>>>
>>>> Details on the proposed work is given at:
>>>>
>>>> https://malhar.atlassian.net/browse/APEX-55 . The work is also related
>>> to
>>>> https://malhar.atlassian.net/browse/APEX-3
>>>>
>>>> Regards,
>>>> -Tushar.
>>>>