fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: [DISCUSS] Improve how Fluo applications are run
Date Wed, 24 May 2017 16:05:19 GMT
There are few possible goals with these changes :

  * support multiple launchers
  * stop using the local app dirs and store all app info in zookeeper and hdfs

Instead of creating a new fluo-local, maybe we could make the
following changes to the current fluo tarball and command.  Maybe
still create a fluo-yarn tarball for launching yarn.

Deprecate the following commands (these commands are yarn and/or app
dir related) :
  fluo list
  fluo new <app name>
  fluo init <app name>
  fluo start <app name>
  fluo stop <app name>
  fluo kill <app name>
  fluo status <app name>
  fluo info <app name>

Add the following commands :

  # sets up a new app, putting its jars in hdfs and config in zookeeper
  fluo setup <app props> {list of app jars}

  # updates a app, putting its jars in hdfs and config in zookeeper
  fluo update <app props>  {list of app jars}

  # retrieves an apps jars from hdfs
  fluo jars <app name> <output dir>

  # retrieves an apps config from zookeeper and prints it stdout
  fluo config <app name>

  # deletes an apps data in zookeeper and hdfs
  fluo delete <app name>

Following commands stay the same :

  # list all apps... or if given an app name lists info about that app
  fluo list [optional app name]
  fluo classpath
  fluo wait <app>
  fluo version
  fluo exec <app> <class>  {<argument>}
  fluo scan <app>

On Fri, May 19, 2017 at 1:18 PM, Mike Walch <mwalch@apache.org> wrote:
> For Fluo 1.2, I would like to improve how Fluo applications are
> launched/run. This involves deprecating the current tarball distribution
> and creating downstream repos that create distributions for running Fluo
> applications locally and cluster managements tools suchs as YARN, Mesos,
> Kubernetes, etc.
>
> I have created repos for running Fluo locally and YARN and have started to
> design their functionality by creating scripts and documentation. No Java
> code has been written yet. The repos will eventually be moved to Apache
> infrastructure.
>
> https://github.com/mikewalch/fluo-local
> https://github.com/mikewalch/fluo-yarn
>
> In order to complete this task, a lot of new functionality/commands need to
> created upstream in Fluo. To coordinate this work, I created several issues
> under the 1.2 milestone.
>
> https://github.com/apache/incubator-fluo/milestone/6
>
> If you are interested in this work, review the documentation/scripts in
> fluo-local & fluo-yarn as well as the issues linked to above and let me
> know if you have any ideas or suggestions.

Mime
View raw message