gearpump-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincent Wang <>
Subject Re: Question about Dynamic Computation DAG
Date Mon, 16 Jan 2017 07:46:52 GMT
Hi Junguk,

  Sorry the missing of documentation. Here is a simple example:

  First, you can submit the *wordcount *application to Gearpump, the
wordcount jar file is in example folder.If everything works fine, you can
see a DAG from Gearpump's dashboard


  After that, you can right click the node and a menu will pop up. The
dynamic DAG now support two operations, *changing the parallelism* and
the entire processor(node)*.
  For replacing the processor, user need to supply an other jar file that
contains the new streaming task's java class.

  Basically every node in a streaming DAG in Gearpump is called a Processor
and it's internal business logic should be implemented by a corresponding
Task by user.
Internally, when replacing a nodeA with nodeB, Gearpump just create a new
nodeB in the original DAG at the right place. And then lead the data stream
to the new nodeB.

  Here are some limitations of dynamic DAG:
  1. Don't support changing DataSource or DataSink
  2. When dynamic DAG happens, at least once is not guaranteed.
  3. Don't support adding a node or deleting a node.


Xu, Qian <>于2017年1月15日周日 下午1:04写道:

> Hi Junguk,
> Thanks for sending me the mail. I'd like you to post your question on the
> user mailing list (
>, so
> that more people might get involved.
> Stanley
> On Sat, Jan 14, 2017 at 3:25 PM, Junguk Cho <> wrote:
> Hi.
> I am new to gearpump.
> While I am reading the document (
> I wonder Dynamic Computation DAG in gearpump. However, the explanation is
> not sufficient.
> Would you explain how it work in details like functions, limitation and
> interface for reconfiguration?
> Thanks,
> Junguk
> --
> Qian Xu (Stanley)
> _______________________________________________________________________
> This e-mail may contain confidential material for the sole use of the
> intended recipient(s). Any review or distribution by others is strictly
> prohibited. If you are not the intended recipient, please contact the
> sender and delete all copies.

View raw message