apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: Megh operator library
Date Sat, 10 Sep 2016 14:04:21 GMT

I understand your concerns but if you look at my original email it
doesn't say to "move around code" but rather to contribute what makes
sense in a "fashion that is consistent with Malhar project".

For some operators that use underlying technologies for which there
are alternate implementations for the underlying technologies already
in Malhar, in a case by case basis we can discuss whether to repurpose
them during contribution or post-contribution.

I want to bootstrap the process. To that end I can come up with an
initial proposal and then we can discuss and modify it as everyone see


> On Sep 9, 2016, at 5:18 PM, Thomas Weise <thomas@datatorrent.com> wrote:
> I see no reason to move the dimension operator along with everything it
> duplicates to Malhar. It's available to use for everyone as it is and there
> should be an initiative to make it confirm to the underlying framework to
> be part of Malhar.
> Also there is already an enrichment operator, there is even documentation
> for it.
> Hence, this needs to be analyzed properly.
> Thomas
> On Fri, Sep 9, 2016 at 5:10 PM, Pramod Immaneni <pramod@datatorrent.com>
> wrote:
>> Yes, I do plan to come up with a proposal with a list. The ones that come
>> to mind are flume, enrichment, various dimensional operators and any custom
>> partitioners. The dimensional operators are in a mature state and usable
>> today, in future they could also be ported onto the new windowing and
>> managed state operator framework.
>> Thanks
>> On Fri, Sep 9, 2016 at 4:29 PM, Thomas Weise <thomas@datatorrent.com>
>> wrote:
>>> A cursory look suggests there is a lot of overlap. I'm looking forward to
>>> see a proposal that reflects a vision how to evolve Malhar rather than
>> just
>>> moving around code.
>>> Thomas
>>> On Thu, Sep 8, 2016 at 2:40 PM, Pramod Immaneni <pramod@datatorrent.com>
>>> wrote:
>>>> Hi,
>>>> DataTorrent, the initial contributor to Apex and the company I work
>> for,
>>>> has opened up a library of operators called Megh recently to the public
>>> and
>>>> has made the repository available under the Apache License. The link to
>>> the
>>>> repository is below. These operators, for the most part, contain
>>>> functionality that is complementary to what Malhar library provides and
>>>> were developed to solve business use cases that arose over time. Also,
>>> some
>>>> operators in Malhar were inspired from early implementations in the
>> Megh
>>>> library and were built upon knowledge gained in doing the original
>>>> implementations.
>>>> Our goal is to not have Megh as a separate library but rather bring
>> these
>>>> operators into Malhar in a fashion that it is consistent with the
>> Malhar
>>>> project and repository. In the upcoming days, in a gradual fashion, we
>>> will
>>>> have more details on the individual operators that we would like to
>>>> contribute. Also, if you are interested in helping with this effort
>>> please
>>>> raise your hand.
>>>> https://github.com/DataTorrent/Megh/
>>>> Thanks

View raw message