gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manu Zhang <owenzhang1...@gmail.com>
Subject Re: Propose to combine module gearpump-daemon into gearpump-core
Date Tue, 11 Oct 2016 05:04:40 GMT
+1.  Integration tests for Beam Gearpump runner heavily depends on
EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
run gearpump examples.

Thanks,
Manu

On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang <fvunicorn@gmail.com> wrote:

> Hi all,
>
>   Currently we have an EmbeddedCluster in gearpump-daemon module which
> allows user to run their gearpump application in a local JVM.
>   Now suppose user want to implement an streaming application using
> Gearpump and use the EmbeddedCluster to test it locally or even in an IDE.
> So the application needs to have extra dependency on the gearpump-daemon
> module to access EmbeddedCluster,  and user have to write code like
> *     if (debugMode) {*
> *      localCluster = new EmbeddedCluster(akkaConf);*
> *      localCluster.start();*
> *    }*
> *    ClientContext masterClient = null;*
> *    if (localCluster != null) {*
> *      masterClient = localCluster.newClientContext();*
> *    } else {*
> *      masterClient = new ClientContext(akkaConf);*
> *    }*
> *    masterClient.submit(app);*
>
>   Which is obviously tedious and inconvenient.
>
>   So we'd like to provide user with a implementation just like what Flink
> did.  Basically we want to integrate EmbeddedCluster into ClientContext so
> that user can switch the application running mode easily by some
> configuration or decision made by ClientConext automatically.
>   However, currently the ClientContext is in module gearpump-core and
> EmbeddedCluster
> has dependencies on Worker and Master daemon actors, which means these
> classes need to be moved along with EmbeddedCluster and finally almost the
> whole gearpump-daemon module need to be integrated into gearpump-core
> module.
>    That's the side effect of achieving a more powerful ClientContext
> integrated with a EmbeddedCluster and I'd like to hear your opinions about
> on this proposal.
>
> Thanks,
> Huafeng
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message