heron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Ramasamy <kart...@streaml.io>
Subject Proposal for Heron API Server
Date Mon, 24 Jul 2017 23:57:12 GMT
*Rationale*:

Currently, Heron supports a single mode of deployment called library mode.
Library mode requires several steps and client side configuration which
could be intensive. Hence, we want to support another mode called service
mode for simplified deployment.

*Library Mode:*

With Heron, the current mode of deployment is called library mode. This
mode does not require any services running for Heron to deploy which is a
huge advantage. However, it requires several configuration to be in the
client side. Because of this administering becomes harder - especially
maintaining the configuration and distributing them when the configuration
is changed. While this is possible for a bigger teams with dedicated
dev-ops team, it might be overhead for medium and smaller teams.
Furthermore, this mode of deployment does not have an API to
submit/kill/activate/deactivate programmatically.

*Service Mode:*

In this mode, an api server will be running as a service. This service will
be run as yet another job in the scheduler so that it will be restarted
during machine and process failures thereby providing fault tolerance. This
api server will maintain the configuration and heron cli will be augmented
to use the rest API to submit/kill/activate/deactivate the topologies in
this mode. The advantage of this mode is it simplifies deployment but
requires running a service.

*Merging Tracker into API Server:*

Current, Heron tracker written in python duplicates the state manager code
in python as well. The API server will support the heron tracker api in
addition to topologies api. Depending on the mode of the deployment, the
api server can be deployed in one of the modes - library mode (which
exposes only the tracker API) and services mode (which exposes both the
tracker + api server). Initially, the tracker and api server will be in
separate directory until great amount of testing is done. Once it is
completed, we can think about cutting over to entirely using API server.

This change will not affect any of the existing deployments and it will be
backward compatible.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message