apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shunxin Lu <lushun...@gmail.com>
Subject Re: Join Support
Date Tue, 16 Aug 2016 20:19:42 GMT
Hi David,

Thanks for the reply. I think I will need to reconsider the whole situation
again base on your input.
The main problem that I had was, every input port has its own type, how can
I write methods that can handle all of them?

Thanks,
Shunxin

On Tue, Aug 16, 2016 at 12:49 PM, David Yan <david@datatorrent.com> wrote:

> Also, regarding the difficulties you mentioned about a new subclass
> inheriting AbstractWindowedOperator, what specifically are they?
>
> David
>
> On Tue, Aug 16, 2016 at 12:31 PM, David Yan <david@datatorrent.com> wrote:
>
> > Hi Shunxin,
> >
> > One problem with join support using WindowedOperator is that Apex
> operator
> > does not support variable number of ports so we might have to limit the
> > join operator to, say, 5 input ports. Implementing join support for
> > WindowedOperator should not be difficult, but might be a little messy
> > because we will need to have a watermark control port for each regular
> > input port, making it 10 total input ports if we support a maximum of 5
> > join inputs.
> >
> > Please take a look at the JoinAccumulation template interface. That was
> > there for the future join support I planned to add.
> >
> > Also, pay a bit of attention on how you process watermarks from each
> > input, and let me know if you need help.
> >
> > David
> >
> > On Fri, Aug 12, 2016 at 11:03 AM, Shunxin Lu <lushunxin@gmail.com>
> wrote:
> >
> >> Hello there,
> >>
> >> I am planning to add join support in Windowed Operator, but need some
> >> advice on how to start.
> >> Currently I am thinking to add a new subclass inheriting
> >> AbstractWindowedOperator and do all the work we need in that class (add
> >> more input ports, do join accumulation, etc.), but I am experiencing
> some
> >> difficulties doing so. Or should I directly change the codes in
> >> AbstractWindowedOperator?
> >>
> >> Thanks,
> >> Shunxin
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message