flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: [Delta Iterations] The dirty insides(insights)
Date Thu, 19 Mar 2015 08:38:56 GMT
Did you send an empty email to user-subscribe@flink.apache.org ? That
should subscribe you.

On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lungu.andra@gmail.com> wrote:

> Hello,
>
> I've used delta iterations several times up until now, but I just realized
> that I never fully understood what happens inside. And the documentation
> only explains things from a user's perspective. Which is why I could really
> use your help :).
>
> Here goes nothing:
> In Gelly, Graph.java, there is a nice runVertexCentricIteration(...)
> method, which is in fact Spargel's - that disguises a delta iteration. What
> I am trying to do is to set the vertex value before running
>
> DataSet<Vertex<K, VV>> newVertices =
> verticesWithInDegrees.runOperation(iteration);
>
> The problem is that after this runOperation, the vertex values get reset.
> Now, when I looked in VertexCentricIteration.java's createResult().
>
> It's a plain delta iteration that (more or less) looks like this:
>
> final DeltaIteration<Vertex<VertexKey, VertexValue>, Vertex<VertexKey,
> VertexValue>> iteration =
>    this.initialVertices.iterateDelta(this.initialVertices,
> this.maximumNumberOfIterations, zeroKeyPos);
>
> ................
>
> // configure coGroup update function with name and broadcast variables
> updates = updates.name("Vertex State Updates");
> for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
>    updates = updates.withBroadcastSet(e.f1, e.f0);
> }
>
> // let the operator know that we preserve the key field
> updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
>
> return iteration.closeWith(updates, updates);
>
> The
>
> DataSet<?>
>
> in the for is the problem. Vertex values get reset. Can someone give me a
> hint on how to propagate the vertex value throughout the iteration?
>
> Thanks!
> Andra
>
> ----------------------------------------------------------------------------------------
> P.S. Could someone please subscribe me to the @user mailing list? For some
> reason, sending the mail to that recipient fails. Thanks!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message