flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lin Li <lincoln.8...@gmail.com>
Subject Re: [DISCUSS] Should we supply a new Iterator instance for Functions with Iterable input(s) like CoGroupFunction ?
Date Wed, 22 Feb 2017 13:28:51 GMT
I created a jira https://issues.apache.org/jira/browse/FLINK-5883, and will
work on this asap.

2017-02-22 21:01 GMT+08:00 Aljoscha Krettek <aljoscha@apache.org>:

> I think this was mostly an oversight on my part that was possible because
> we didn't have good test-coverage that was enforcing correctness. Please go
> ahead and open an issue for re-adding the throw.
>
> On Wed, 22 Feb 2017 at 13:28 Lin Li <lincoln.86xy@gmail.com> wrote:
>
> > Thank you for the answer!
> >
> > The discussion on FLINK-1023 is very clear to me. I agree with that
> throws
> > a TraversableOnceException when the iterator is requested the second
> time.
> >
> > @Aljoscha git history shows you removed the exception-thrown code from
> > FLINK-1110, would you mind me create an issue and add it back?
> >
> > BTW, I had submitted a pr for FLINK-5498 (
> > https://github.com/apache/flink/pull/3379), support left/right outer
> joins
> > with non-equi-join conditions via coGroup operator with a generated
> > OuterJoinCoGroupFunction.
> > But current implementation is not memory safe when do a many-to-one/many
> > outer join which will copy the opposite side input into an List
> buffer(This
> > is not pretty though it follows the guideline, just remember the input
> data
> > within a function call). It's a work-around for now, in the long run, I
> > think we should extend the runtime join operators to support such
> > non-equi-join conditions.  Implementation in TableAPI layer could not
> >  always ensures the efficiency.
> > Welcome any suggestions on current solution.
> >
> > Best, Lincoln
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message