gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiang Weihua <whji...@outlook.com>
Subject Re: Propose to combine module gearpump-daemon into gearpump-core
Date Wed, 12 Oct 2016 01:22:37 GMT
+1 as I can’t find another better solution.

在 2016/10/11 下午4:29,“Karol Brejna”<karol.brejna@gmail.com> 写入:

    +1
    I think moving EmbeddedCluster capabilities to gearpump-core makes
    sense and simplifies things.
    
    On Tue, Oct 11, 2016 at 7:04 AM, Manu Zhang <owenzhang1990@gmail.com> wrote:
    > +1.  Integration tests for Beam Gearpump runner heavily depends on
    > EmbeddedCluster. It's quite weird to add `gearpump-daemon` dependency to
    > run gearpump examples.
    >
    > Thanks,
    > Manu
    >
    > On Tue, Oct 11, 2016 at 10:17 AM Vincent Wang <fvunicorn@gmail.com> wrote:
    >
    >> Hi all,
    >>
    >>   Currently we have an EmbeddedCluster in gearpump-daemon module which
    >> allows user to run their gearpump application in a local JVM.
    >>   Now suppose user want to implement an streaming application using
    >> Gearpump and use the EmbeddedCluster to test it locally or even in an IDE.
    >> So the application needs to have extra dependency on the gearpump-daemon
    >> module to access EmbeddedCluster,  and user have to write code like
    >> *     if (debugMode) {*
    >> *      localCluster = new EmbeddedCluster(akkaConf);*
    >> *      localCluster.start();*
    >> *    }*
    >> *    ClientContext masterClient = null;*
    >> *    if (localCluster != null) {*
    >> *      masterClient = localCluster.newClientContext();*
    >> *    } else {*
    >> *      masterClient = new ClientContext(akkaConf);*
    >> *    }*
    >> *    masterClient.submit(app);*
    >>
    >>   Which is obviously tedious and inconvenient.
    >>
    >>   So we'd like to provide user with a implementation just like what Flink
    >> did.  Basically we want to integrate EmbeddedCluster into ClientContext so
    >> that user can switch the application running mode easily by some
    >> configuration or decision made by ClientConext automatically.
    >>   However, currently the ClientContext is in module gearpump-core and
    >> EmbeddedCluster
    >> has dependencies on Worker and Master daemon actors, which means these
    >> classes need to be moved along with EmbeddedCluster and finally almost the
    >> whole gearpump-daemon module need to be integrated into gearpump-core
    >> module.
    >>    That's the side effect of achieving a more powerful ClientContext
    >> integrated with a EmbeddedCluster and I'd like to hear your opinions about
    >> on this proposal.
    >>
    >> Thanks,
    >> Huafeng
    >>
    


Mime
View raw message