incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "P. Taylor Goetz" <ptgo...@gmail.com>
Subject Re: [PROPOSAL] Gossip incubator proposal
Date Tue, 19 Apr 2016 00:52:34 GMT
Thanks for the input Henry!

I've seen proposals in the past come with what seems like a fully-baked list of mentors, and
others with a short list that could obviously benefit from having more mentors and are looking
for volunteers.

I think Gossip falls into the latter category and would welcome volunteer mentors. So consider
this a call for additional mentors.

In recent incubator discussions there's been some debate about the role of champion and how
it relates to mentoring. I'm personally not sure where I stand in that debate, and so have
not commented. I accepted the role of champion with the intent of also acting as a mentor
going forward.

Is that (champion also acting as a mentor) something that we (IPMC) think should be explicitly
called out in the proposal?

-Taylor

> On Apr 18, 2016, at 8:35 PM, Henry Saputra <henry.saputra@gmail.com> wrote:
> 
> I would recommend getting more mentors to help. Especially almost half of
> initial committers are not from Apache and the other half seemed very busy
> with other projects.
> 
> - Henry
> 
>> On Mon, Apr 18, 2016 at 12:42 PM, P. Taylor Goetz <ptgoetz@gmail.com> wrote:
>> 
>> Bump.
>> 
>> Are there any further comments/questions on the proposal? Or should we
>> proceed with a VOTE?
>> 
>> -Taylor
>> 
>> 
>>> On Apr 13, 2016, at 6:58 PM, Edward Capriolo <edlinuxguru@gmail.com>
>> wrote:
>>> 
>>> Hello all,
>>> 
>>> I would like to open up a discussion on the following proposal:
>>> https://wiki.apache.org/incubator/GossipProposal
>>> 
>>> Besides those  mentioned in the proposal, thank you to Jake Ferrel, Lewis
>>> McGibbne, and Nate McCall who were a tremendous help!
>>> 
>>> A copy of the proposal is below:
>>> 
>>> = Abstract =
>>> 
>>> Apache Gossip will be an implementation of the Gossip Protocol based
>>> on code available here: https://github.com/edwardcapriolo/gossip/
>>> which is already licenced using the glorious Apache V2 License.
>>> 
>>> = Proposal =
>>> 
>>> Apache Gossip aims to provide a gossip based consensus protocol
>>> written in Java for peer-to-peer communication to the Apache Incubator
>>> (http://incubator.apache.org/). This implementation will effectively
>>> scale from one to one-thousand node clusters. In addition to the code
>>> implementation, the project should produce specifications of the wire
>>> protocol, features, and expected behavior of the system such that
>>> compatible implementations can communicate.
>>> 
>>> = Background =
>>> 
>>> The gossip protocol has been implemented to varying levels of rigor by
>>> a number of entities. In particular, Apache Cassandra uses an
>>> implementation of gossip to locate peers and transmit up/down state.
>>> Apache Spark leverages tooling in Akka which provides peer-to-peer
>>> node discovery capabilities.
>>> 
>>> *
>> http://highscalability.com/blog/2011/11/14/using-gossip-protocols-for-failure-detection-monitoring-mess.html
>>> 
>>> * https://en.wikipedia.org/wiki/Gossip_protocol
>>> 
>>> = Rationale =
>>> 
>>> With distributed computing becoming extremely widespread, and the
>>> growth of the buzz-factor of ‘the-internet-of-things’ it is
>>> increasingly important that networks of IP addressable devices can
>>> form a peer-to-peer network. Applications of peer-to-peer networks
>>> include generating crypto currency, managing hardware such as solar
>>> power micro-grids, and more traditional roles like grid/High
>>> Performance Computing and distributed storage systems. Different
>>> implements of gossip based consensus protocols have been implemented
>>> in numerous languages or as part of more complex software stacks. The
>>> Apache foundation should lead the effort of producing a purpose built
>>> tool that can be used by downstream projects to form peer-to-peer
>>> networks.
>>> 
>>> = Initial Goals =
>>> 
>>> * Migration of current code https://github.com/edwardcapriolo/gossip
>>> to the Apache Software Foundation
>>> * Secure communications
>>> * Transport security using a pre-shared key
>>> * Public Key Infrastructure
>>> * Introduce a cluster name to wire protocol to avoid misconfigurations
>>> * Effectively operate when systems have multiple network interfaces
>>> by controlling IP binding settings
>>> * Effectively operate when systems have Network Address Translations
>>> devices between them using a broadcast IP settings
>>> * Develop advanced integration testing from cluster sizes of 1-1000 nodes
>>> * Test convergence times
>>> * Demonstrate the tradeoffs of different settings in regard to
>>> bandwidth/cpu/convergence time/accuracy
>>> * Gossip data other than cluster state such as application/user data
>>> * Provide detailed specifications such that others can implement the
>>> protocol in other programming languages
>>> * Explore HTTP transport as an alternative to UDP
>>> 
>>> = Current Status =
>>> 
>>> The current code has been around for some time. Previously it was a
>>> google code project. Since the fork in January 2015 there have been 55
>>> commits and 4 releases.
>>> 
>>> == Meritocracy ==
>>> 
>>> We believe in meritocracy. All suggestions are taken seriously. We
>>> enjoy helping new people become part of process. For other projects
>>> available on our github, once a user shows enough activity we grant
>>> them collaborator status.
>>> 
>>> == Community ==
>>> 
>>> In a relatively short amount of time, with a small amount of promotion
>>> on twitter and through blogging, we have 50+ followers on github and
>>> several forks of the project.. With the Apache brand we should be able
>>> to attract more. Once we have entered the incubator we believe it will
>>> be easier to attempt to unify with other similar implementations.
>>> 
>>> == Core Developers ==
>>> 
>>> The code was forked on Jan 9th 2015, since then there have been 4
>>> releases and 55 commits. The majority of th work is by Edward
>>> Capriolo. Several people are interested in the features of this
>>> proposal and have indicated they will volunteer their time.
>>> 
>>> == Alignment ==
>>> 
>>> Apache is the perfect organization to take on this Gossip project.
>>> Besides benefiting a number of projects directly, the active
>>> development and reach will help Gossip be adopted and a leader in the
>>> space.
>>> 
>>> = Known Risks =
>>> 
>>> Several existing implementations of similar cluster membership systems
>>> (gossip based and otherwise) exist. A key challenge is moving from a
>>> relatively niche technical audience to a more general tool for solving
>>> a common problem. Differentiating when Apache Gossip may be the
>>> optimal solution versus a clouded landscape of coordination services
>>> such as etcd or zookeeper, and distributed data stores via feature set
>>> will also be key. We believe that users be attracted to the
>>> peer-to-peer distributed service toolkit that gossip will provide.
>>> 
>>> == Orphaned products ==
>>> 
>>> We plan on building on the current code by developing discrete
>>> features with a focus on testing. Up until this point the project has
>>> been maintained be a single person. However the project currently
>>> releases artifacts to maven central, is tested using travis CI, and
>>> follows controlled development practice. This level of dedication will
>>> see the process through the initial stages.
>>> 
>>> == Inexperience with Open Source ==
>>> We are very familiar with Open Source development and the Apache
>>> Foundation. The current code base already carries an Apache V2
>>> Licence.
>>> == Homogenous Developers ==
>>> Multiple people have made contributions to the current code base. This
>>> proposal has generated more interest and several more are offering to
>>> volunteer time. These volunteers are from diverse corporate entities.
>>> == Reliance on Salaried Developers ==
>>> We wish to create Apache Gossip for the challenge of producing great
>>> software. Initially all members of the project will volunteer their
>>> time and no one will be expressly salaried to work only on this
>>> project.
>>> 
>>> == Relationships with Other Apache Products ==
>>> If the Apache Gossip project is successful, other products in Apache
>>> such as Storm or Cassandra could adopt it. However, adoption by those
>>> specific projects is not our criteria for success. There are a large
>>> number of applications for this system. One example is Apache JMeter
>>> could be built with Apache Gossip as a backend for distributed
>>> testing. Another example is a polyglot registry of thrift services
>>> with gossip based discovery.
>>> == A Excessive Fascination with the Apache Brand ==
>>> We care about the Apache foundation. Having the recognition of the
>>> Apache incubator will undoubtedly help the project. We do not seek the
>>> Apache brand to be used as legal shield or personal glory. We believe
>>> in the Apache foundation and will manage the project with espirit de
>>> corps, welcoming all through meritocracy while using bylaws as guiding
>>> values.
>>> 
>>> = Initial Source =
>>> The initial source for https://github.com/edwardcapriolo/gossip was
>>> forked from https://code.google.com/p/java-gossip/. The code base
>>> always Apache V2 licensed.
>>> 
>>> = Source and Intellectual Property Submission Plan =
>>> 1. Make final commit on teknek-gossip explaining the move to ASF.
>>> 1. Move code into ASF repo.
>>> 1. Rename references to old name.
>>> 1. Apply Apache V2 licence to all source files.
>>> 
>>> = External Dependencies =
>>> Currently the project encodes messages into JSON for network
>>> transmission. This is done using JSONorg, but will switch to jackson
>>> (potentially before the move to the ASF).
>>> = Cryptography =
>>> The current code is not using cryptography. It is on the road map to
>>> add security through transport encryption (SSL) and transport
>>> encryption.
>>> = Required Resources =
>>> == Mailing lists ==
>>> * dev@gossip.incubator.apache.org
>>> * commits@gossip.incubator.apache.org
>>> * private@gossip.incubator.apache.org
>>> The user list will be added when we have broader adoption.
>>> == Git Repository ==
>> https://git-wip-us.apache.org/repos/asf/incubator-gossip.git
>>> == Issue Tracking ==
>>> JIRA tracker: GOSSIP
>>> = Initial Committers =
>>> * Edward Capriolo (Hive Committer, PMC)
>>> * P. Taylor Goetz (Storm PMC)
>>> * Gary Dusbabek (Cassandra Committer, PMC)
>>> * Dorian Ellerbe (requires CLA)
>>> * Sathish Dhinakaran (requires CLA)
>>> = Affiliations =
>>> With diverse contributors the project will be able to make balanced
>>> decisions best for the future of the project.
>>> = Additional Interested Contributors =
>>> 
>>> Those interested in getting involved with the project as it starts are
>>> encourage to list themselves here.
>>> 
>>> * (next entry here)
>>> 
>>> = Sponsors =
>>> == Champion ==
>>> P. Taylor Goetz has offered to champion.
>>> == Mentors ==
>>> * Sean Busbey (ASF Member, IPMC, various Apache PMCs, YCSB maintainer)
>>> * Josh Elser (ASF Member, IPMC, Apache Accumulo PMC, Apache Calcite PMC)
>>> = Sponsoring Entity =
>>> The Incubator
>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message