hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Motgi <ni...@continuuity.com>
Subject Introducing Weave - Apache YARN simplified.
Date Tue, 28 May 2013 22:21:58 GMT
Hi All,

At Continuuity, we use Apache YARN as an integral part of our products because of its vision
to support a diverse set of applications and processing patterns. One such product is BigFlow,
our realtime distributed stream-processing engine. Apache YARN is used to run and manage BigFlow
applications including lifecycle and runtime elasticity. During our journey with YARN we have
come to the realization that it is extremely powerful but its full capability is challenging
to leverage.  It is difficult to get started, hard to test and debug, and complex to build
new kinds of non-MapReduce applications and frameworks on. 

We decided to build Weave and be part of the journey to take Apache YARN to the next level
of usability and functionality. We have been using Weave extensively to support our products
and have seen the benefit and power of Apache YARN and Weave combined.  We have decided to
share Weave under the Apache 2.0 license in an effort to collaborate with members of the community,
broaden the set of applications and patterns that Weave supports, and further the overall
adoption of Apache YARN. 

Weave is NOT a replacement for Apache YARN.  It is instead a value-added framework that operates
on top of Apache YARN. 

What is Weave ?

Weave is a simple set of libraries that allows you to easily manage distributed applications
through an abstraction layer built on Apache YARN. Weave allows you to use YARN’s distributed
capabilities with a programming model that is similar to running threads. 

Features of Weave

  - Simple API for specifying, running and managing application lifecycle
  - An easy way to communicate with an application or parts of an application
  - A generic Application Master to better support simple applications
  - Simplified archive management and local file transport
  - Improved control over application logs, metrics and errors
  - Discovery service
  - And many more...

Where can I get this coolness ?

The code is available on github at http://github.com/continuuity/weave under the Apache 2.0

We will continue adding features to improve Weave, but would love to hear from people who
are willing to be contributors to this project and help us make it better.  If you are not
interested in contributing - no problem - we would  still love to hear your comments, questions,
and concerns.

Thanks for your time and we look forward to hearing your thoughts on Weave.

The Continuuity Team
View raw message