incubator-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Incubator Wiki] Update of "LivyProposal" by SeanBusbey
Date Fri, 19 May 2017 23:29:33 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Incubator Wiki" for change notification.

The "LivyProposal" page has been changed by SeanBusbey:
https://wiki.apache.org/incubator/LivyProposal?action=diff&rev1=1&rev2=2

Comment:
Missed a footnote.

  == Reliance on salaried Developers ==
  The existing contributors to the Livy project have been made by salaried engineers from
Cloudera, Microsoft and Hortonworks. Since there are three major organisations involved, the
risk of reliance on a single group of salaried developers is mitigated. The Livy user base
is diverse, with users from across the globe, including users from academic settings. We aim
to further diversify the Livy user and contributor base.
  == Relationships with other Apache projects ==
- Livy is closely tied to the Apache Spark project and currently addresses the scenarios for
a REST based batch and interactive gateway for Spark jobs on YARN. Given the growing number
of integrations with Livy, keeping it outside of Apache Spark aligns with the desire of the
Apache Spark community to reduce the number of external dependencies in the Spark project.
Specifically, the Apache Spark community has previously expressed a desire to keep job servers
independent from the project. Furthermore, while Livy common usage is closely tied to Spark
deployments right now, its core building blocks can be reused elsewhere.  Livy’s Remote
REPL could be used as a library for interactive scenarios in non-Spark projects. In the future,
integrations with cluster managers like Apache Mesos and others could also be added.
+ Livy is closely tied to the Apache Spark project and currently addresses the scenarios for
a REST based batch and interactive gateway for Spark jobs on YARN. Given the growing number
of integrations with Livy, keeping it outside of Apache Spark aligns with the desire of the
Apache Spark community to reduce the number of external dependencies in the Spark project.
Specifically, the Apache Spark community has previously expressed a desire to keep job servers
independent from the project.<<FootNote(See, for example, discussion of the Ooyala Spark
Job Server in SPARK-818)>> Furthermore, while Livy common usage is closely tied to Spark
deployments right now, its core building blocks can be reused elsewhere.  Livy’s Remote
REPL could be used as a library for interactive scenarios in non-Spark projects. In the future,
integrations with cluster managers like Apache Mesos and others could also be added.
  
  The features provided by Livy have already been integrated with existing projects like Jupyter
and Apache Zeppelin for their interactive Spark use cases. This validates the need for a project
like Livy and provides an active downstream user base that the Livy community can interact
with to seed future interest in the project.
  

---------------------------------------------------------------------
To unsubscribe, e-mail: cvs-unsubscribe@incubator.apache.org
For additional commands, e-mail: cvs-help@incubator.apache.org


Mime
View raw message