apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vlad Rozov <v.ro...@datatorrent.com>
Subject Re: Modules support in Apex
Date Wed, 02 Sep 2015 22:09:12 GMT
Hi Thomas,

I'll update JIRA after I rebase my work into the apache apex fork.

Thank you,

Vlad

On 9/2/15 14:53, Thomas Weise wrote:
> Vlad,
>
> Can you send a pointer to the fork you are working on (or mention it on the
> JIRA)?
>
> Thanks
>
> On Mon, Aug 31, 2015 at 2:17 PM, Vlad Rozov <v.rozov@datatorrent.com> wrote:
>
>> Atri,
>>
>> As a first cut module is a predefined subgraph that can be inserted into a
>> DAG. Generally speaking module is not required to expand to the same
>> subgraph. Depending on module properties or configuration, module may
>> expand to different subgraphs. At the same time, similar to an operator
>> module has predefined input/output ports and properties.
>>
>> Thank you,
>>
>> Vlad
>>
>>
>> On 8/31/15 11:46, Atri Sharma wrote:
>>
>>> No, I dont think APEX-3's functionality needs to be exceeded for this.
>>> What
>>> I am trying to understand here is the concept of Module. Is it a family of
>>> operators defined by same interface implementation *or* is it a defined
>>> subgraph that can be replaced and used as a part of a subgraph instead of
>>> building the whole connection again?
>>>
>>> On Mon, Aug 31, 2015 at 11:27 PM, Tushar Gosavi <tushar@datatorrent.com>
>>> wrote:
>>>
>>> Yes, you are correct. The APEX-3 is for dynamic expansion of the DAG, If
>>>> we
>>>> can
>>>> expand the DAG dynamically then we can use that functionality to expand
>>>> the
>>>> DAG
>>>> with know list of operator and connections between them. In a way APEX-3
>>>> provides
>>>> bigger functionality than APEX-55.
>>>>
>>>> Let me know if you think that module functionality requires more support
>>>> that what will be provided by APEX-3.
>>>>
>>>>
>>>>
>>>> On Mon, Aug 31, 2015 at 10:39 AM, Atri Sharma <atri@apache.org> wrote:
>>>>
>>>> Looks interesting.
>>>>> Can you explain a bit on how it helps APEX-3? The objective of APEX-3
is
>>>>>
>>>> to
>>>>
>>>>> have dynamic expansion of sub DAGs. If I understand correctly, the
>>>>> static
>>>>> list is to be known when declaring a module.
>>>>>
>>>>> Please correct me if I am wrong.
>>>>>
>>>>> On Mon, Aug 31, 2015 at 11:05 PM, Tushar Gosavi <tushar@datatorrent.com
>>>>> wrote:
>>>>>
>>>>> Hi All,
>>>>>> We are working on adding support for Modules in Apex. A module is
a
>>>>>>
>>>>> group
>>>>> of operators that will have their own existence and will ease the way
>>>>> we
>>>>> are currently defining an application.
>>>>>> A module is defined by:
>>>>>>
>>>>>>      - list of operators.
>>>>>>      - list of input and output ports.
>>>>>>      - set of properties for the module.
>>>>>>      - set of attributes for the module.
>>>>>>
>>>>>>
>>>>>> Details on the proposed work is given at:
>>>>>>
>>>>>> https://malhar.atlassian.net/browse/APEX-55 . The work is also related
>>>>>>
>>>>> to
>>>>>
>>>>>> https://malhar.atlassian.net/browse/APEX-3
>>>>>>
>>>>>> Regards,
>>>>>> -Tushar.
>>>>>>
>>>>>>


Mime
View raw message