flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: cogroup
Date Mon, 29 Jun 2015 16:22:08 GMT
Yes, if you need outer join semantics you have to go with CoGroup.
Some members of the Flink community are working on true outer joins for
Flink, but I don't know what the progress is.

Best, Fabian

2015-06-29 18:05 GMT+02:00 Michele Bertoni <michele1.bertoni@mail.polimi.it>
:

>  thanks both for answering,
> that’s what i expected
>
>  I was using join at first but sadly i had to move from join to cogroup
> because I need outer join
>
>  the alternative to the cogroup is to “complete” the inner join
> extracting from the original dataset what did not matched in the cogroup by
> difference but I don’t think it is convenient
>
>
>
>
>  Il giorno 29/giu/2015, alle ore 17:58, Fabian Hueske <fhueske@gmail.com>
> ha scritto:
>
>   If you just want to do the pairwise comparison try join().
> Join is an inner join and will give you all pairs of elements with
> matching keys.
>  For CoGroup, there is no other way than collecting one side in memory.
>
>  Best, Fabian
>
> 2015-06-29 17:42 GMT+02:00 Matthias J. Sax <mjsax@informatik.hu-berlin.de>
> :
>
>> Why do you not use a join? CoGroup seems not to be the right operator.
>>
>> -Matthias
>>
>> On 06/29/2015 05:40 PM, Michele Bertoni wrote:
>> > Hi I have a question on cogroup
>> >
>> > when I cogroup two dataset is there a way to compare each element on
>> the left with each element on the right (inside a group) without collecting
>> one side?
>> >
>> > right now I am doing
>> >
>> > left.cogroup(right).where(0,1,2).equalTo(0,1,2){
>> >       (leftIterator, rightIterator, out) => {
>> >               val lSet = leftIterator.toSet           // <———— toSet
>> >               for(r <- rightIterator)
>> >                       for(l <- lSet)
>> >                               //do something
>> >       }
>> > }
>> >
>> > I would like to avoid the toSet
>> >
>> >
>> > thanks for help
>> >
>>
>>
>
>

Mime
View raw message