incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@hortonworks.com>
Subject Re: [VOTE] Accept Helix into Apache Incubator
Date Wed, 10 Oct 2012 20:33:27 GMT
+1 binding.

On Wed, Oct 10, 2012 at 1:32 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
> +1 (binding)
>
> On Wed, Oct 10, 2012 at 9:37 AM, kishore g <g.kishore@gmail.com> wrote:
>
>> Hi,
>>
>> I would like to call a vote for accepting Helix for incubation in the
>> Apache Incubator. I have pasted the full proposal below.
>>
>> Please cast your vote:
>>
>> [ ] +1, bring Helix into Incubator
>> [ ] +0, I don't care either way,
>> [ ] -1, do not bring Helix into Incubator, because ...
>>
>> This vote will be open for 72 hours and only votes from the Incubator
>> PMC are binding.
>>
>> Thanks,
>> Kishore G
>>
>>
>> == Abstract ==
>> Helix is a cluster management system for managing partitioned and
>> replicated resources in distributed data systems.
>>
>> == Proposal ==
>> Helix provides an abstraction that separates coordination and
>> management tasks from functional tasks of a distributed system. The
>> developer defines the system behavior via a state machine, the
>> transitions between those states, and constraints on states and
>> transitions that govern the system’s valid settings. Helix ensures the
>> distributed system satisfies the state machine, controlling state
>> changes as appropriate during common operational activities such as
>> upgrades, component failures, bootstrapping, running maintenance
>> tasks, and adding capacity.
>>
>> == Background ==
>> Helix was developed at LinkedIn to manage large clusters for several
>> diverse applications, including a distributed, partitioned,
>> replicated, highly available document store with a master-slave model,
>> a search service with multiple replicas that are updated atomically
>> and in near real-time, and a change data capture service for reliably
>> transporting database changes to caches, other dependent databases and
>> indexes.
>>
>> These services use Helix to reliably manage dozens of clusters in
>> multiple data centers.  These services meet stringent SLAs at large
>> scale for mission-critical production applications such as search,
>> social gestures, and profiles.
>> Helix has proven to be flexible for a wide variety of system
>> configurations and operational patterns, is easy to integrate, with
>> pluggable interfaces enabling custom behavior.  It depends on Apache
>> Zookeeper for coordination and tracking of system state across the
>> cluster, as well as providing fault tolerance.
>> Helix is written in Java. It was developed internally at LinkedIn to
>> meet our particular use cases, but will be useful to many
>> organizations facing a similar need to manage large clusters.
>> Therefore, we would like to share it the ASF and begin developing a
>> community of developers and users within Apache.
>>
>> == Rationale ==
>> Many organizations can benefit from a generalized cluster management
>> system such as Helix. While our distributed data systems use-cases for
>> a very large website like LinkedIn has driven the design of Helix, its
>> uses are varied and we expect many new use cases to emerge.
>>
>> == Current Status ==
>> === Meritocracy ===
>> Our intent with this incubator proposal is to start building a diverse
>> developer community around Helix following the Apache meritocracy
>> model. Since Helix was initially developed in late 2011, we have had
>> fast adoption and contributions by multiple teams at LinkedIn.
>> We plan to continue support for new contributors and work with those
>> who contribute significantly to the project to make them committers.
>>
>> === Community ===
>> Helix is currently being used internally at LinkedIn and is in
>> production in that company for customer-facing features. Recent public
>> presentations of Helix and its goals garnered much interest from
>> potential contributors. We hope to extend our contributor base
>> significantly and invite all those who are interested in building
>> large-scale distributed systems to participate.
>> To further this goal, we use GitHub issue tracking and branching
>> facilities.
>>
>> === Core Developers ===
>> Helix is currently being developed by three engineers at LinkedIn:
>> Kishore Gopalakrishna, Shi Lu and Jason Zheng, and Adam Silberstein,
>> an engineer at Trifacta.  Kishore, the lead developer and architect,
>> has experience within Apache as an S4 committer. Shi developed the
>> partition to node mapping and rebalancing algorithm, cluster admin
>> APIs, and the health check framework.  Jason developed the cluster
>> controller and most of the test framework.  Adam developed the rich
>> alerting framework that enables cluster-wide, “intelligent“ alerts.
>>
>> === Alignment ===
>> The ASF is the natural choice to host the Helix project as its goal of
>> encouraging community-driven open-source projects fits with our vision
>> for Helix. Many projects that can benefit from Helix will rely on
>> Apache ZooKeeper for cluster state management, and can far more easily
>> achieve their operational goals by using Helix.
>>
>> == Known Risks ==
>> === Orphaned Products ===
>> The core developers plan to work full time on the project. There is
>> very little risk of Helix being abandoned as it is a critical part of
>> LinkedIn's internal infrastructure and is in production use.
>>
>> === Inexperience with Open Source ===
>> Only one of the core developers has experience with open source
>> development. Kishore has been actively involved with the ASF as a
>> committer and lead developer of S4.
>>
>> === Homogeneous Developers ===
>> The current core developers are all from LinkedIn. However, we hope to
>> establish a developer community that includes contributors from
>> several corporations and we are actively encouraging new contributors
>> via the mailing lists and public presentations of Helix.
>>
>> === Reliance on Salaried Developers ===
>> Currently, the developers are paid to do work on Helix. However, once
>> the project has a community built around it, we expect to get
>> committers, developers and community from outside the current core
>> developers. However, because LinkedIn relies on Helix internally, the
>> reliance on salaried developers is unlikely to change.
>>
>> === Relationships with Other Apache Products ===
>> Helix uses Apache ZooKeeper to coordinate its state amongst the
>> managed cluster components and for leader election to provide fault
>> tolerance, and uses Apache Maven for build management.
>>
>> === An Excessive Fascination with the Apache Brand ===
>> While we respect the reputation of the Apache brand and have no doubts
>> that it will attract contributors and users, our interest is primarily
>> to give Helix a solid home as an open source project following an
>> established development model. We have also given reasons in the
>> Rationale and Alignment sections.
>>
>> == Documentation ==
>> Information about Helix can be found at
>> [https://github.com/linkedin/helix/wiki]. The following links provide
>> more information about the project:
>>
>>  * Home Page: http://linkedin.github.com/helix/
>>  * Github source: https://github.com/linkedin/helix
>>  * Documentation: https://github.com/linkedin/helix/wiki
>>  * Javadocs: http://linkedin.github.com/helix/apidocs/
>>
>> == Initial Source ==
>> Helix has been under development at LinkedIn since April 2011. It is
>> currently hosted on github under the Apache license 2 at
>> [https://github.com/linkedin/helix]
>> Helix is written in Java. Its source tree is entirely self-contained
>> and relies on Maven as its build system and dependency resolution
>> mechanism.
>>
>> == External Dependencies ==
>> The dependencies all have Apache compatible licenses.
>>  * log4j
>>  * zookeeper
>>  * xstream
>>  * jackson-core-asl
>>  * jackson-mapper-asl
>>  * commons-io
>>  * commons-cli
>>  * commons-math
>>  * zkclient
>>  * camel-josql
>>  * camel-core
>>  * gentlyweb-utils
>>  * josql
>>  * commons-management
>>  * commons-logging-api
>>  * org.restlet
>>  * com.noelios.restlet
>>  * net.sf.jsqlparser
>>
>> Non-Apache build tools that are used by Helix are as follows:
>>  * Cobertura: GNU GPLv2
>> Note that Cobertura is optional and is only used for calculating unit
>> test coverage.
>>
>>
>> == Cryptography ==
>> Not applicable.
>>
>> == Required Resources ==
>> === Mailing Lists ===
>>  * helix-private for private PMC discussions (with moderated subscriptions)
>>  * helix-dev
>>  * helix-commits
>>  * helix-user
>>
>> === Git Directory ===
>> Since Git is now available to be used as primary repo type, Helix
>> would be available in the git repository instead of svn.
>> [https://git.apache.org/helix.git]
>>
>> === Issue Tracking ===
>> JIRA Helix (HELIX)
>>
>> === Other Resources ===
>> The existing code already has unit and integration tests, so we would
>> like a Jenkins instance to run them whenever a new patch is submitted.
>> This can be added after project creation.
>>
>> == Initial Committers ==
>>  * Kishore Gopalakrishna
>>  * Shi Lu
>>  * Zhen Zheng
>>  * Adam Silberstein
>>  * Kapil Surlaker
>>  * Bob Schulman
>>  * Swaroop Jagadish
>>  * Rahul Aggarwal
>>  * Terence Yim
>>  * Santiago Perez
>>
>>
>> == Affiliations ==
>>  * Kishore Gopalakrishna (LinkedIn)
>>  * Shi Lu (LinkedIn)
>>  * Jason Zheng (LinkedIn)
>>  * Adam Silberstein (Trifacta)
>>  * Kapil Surlaker (LinkedIn)
>>  * Bob Schulman (LinkedIn)
>>  * Swaroop Jagadish (LinkedIn)
>>  * Rahul Aggarwal (LinkedIn)
>>  * Terence Yim (LinkedIn)
>>  * Santiago Perez (LinkedIn)
>>
>> == Sponsors ==
>> === Champion ===
>>  * Patrick Hunt (Apache Member)
>>
>> === Nominated Mentors ===
>>  * Patrick Hunt  (Apache Member)
>>  * Mahadev Konar (Apache Member)
>>  * Owen O'Malley (Apache Member)
>>
>> === Sponsoring Entity ===
>> We are requesting the Incubator to sponsor this project.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message