Hi,

you could apply a filter operation = after the cross operation which filters all combinations out which are not = in ascending order.

Cheers,

Till

On Sun, Feb 8, 2015 at 12:38 PM, tanguy racinet wrote:
H= i,

Thank you for you reply. It helped us solve the loopi= ng problems in a nicer way.

We are struggling with s= ome aspects of the cross function.
Still trying to implement the = Apriori algorithm, we need to create combinations of frequent itemSets.
Our problem is that the crossing gives us duplicates, for instance := (1, 2, 3, 4) and (2, 1, 4, 3) are equivalent for us so we are trying to fin= d a way to remove that kind of duplicate in our DataSet.

We already removed duplicates inside our combinations (1, 1, 2) =3D&= gt; (1, 2).

We were thinking about using HashSet b= ut they are not serializable and we cannot use them inside the workflow, bu= t only inside functions.

Can you think of any way = to remove those duplicates ?

Thank you,
=E1=90=A7

=
 =C2=A0 Raci= net TanguyEIT ICT Labs=C2=A0Master Sc= hool StudentDistributed Systems and ServicesTel : +33 6 63 20 89 16=C2=A0/ +49 176 3749 8854Mail : tanracinet@gmail.com
<= /div>

On Thu, Feb 5,= 2015 at 8:51 PM, Vasiliki Kalavri <vasilikikalavri@gmail.com&= gt; wrote:
Hi,

I'm not familiar with the particular algorithm= , but you can most probably use one of the two iterate operators in Flink.<= /div>

Yo= u can read a description and see some examples in the documentation:=C2=A0<= /div>=

Let u= s know if you have any questions!

<= /div>
Cheers,
V.

On 5 February 2015 at 20:37, tanguy racinet <<= a href=3D"mailto:tanracinet@gmail.com" target=3D"_blank">tanracinet@gmail.c= om> wrote:
Hi,

We are = trying to develop the Apriori algorith with the Flink for our Data minning = project.
In our understanding, Flink could handle loop within the= workflow.
However, our knowledge is limited and we cannot find a= nice way to do it.

Here is the flow of my algorit= hm :
GenerateCandidates ----> CalculateFrequentItemSet
mapper =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0----> reducer

We would like to use= the reducer result as the mapper's entry for a predefined number of ti= mes (loop x times).

Is there any smart way to that= with Flink. Or should we just copy paste the loop x times ?

=
Thank you,
 =C2=A0 Raci= net TanguyEIT ICT Labs=C2=A0Master Sc= hool StudentDistributed Systems and ServicesTel : +33 6 63 20 89 16=C2=A0/ +49 176 3749 8854Mail : tanracinet@gmail.com
<= /div>

=E1=90= =A7