incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: [PROPOSAL] Gossip incubator proposal
Date Tue, 19 Apr 2016 00:59:34 GMT
In the Quickstep proposal, Roman explicitly listed himself as both Champion and Mentor. I think
that is a good model to follow. If someone lists themselves only as Champion would I assume
that their responsibilities end shortly after incubation starts.

To use the “it takes a village” metaphor, I think that incubator projects really need
one or two committed parents rather than lots and lots of part-time aunts and uncles.

Julian


> On Apr 18, 2016, at 5:52 PM, P. Taylor Goetz <ptgoetz@gmail.com> wrote:
> 
> Thanks for the input Henry!
> 
> I've seen proposals in the past come with what seems like a fully-baked list of mentors,
and others with a short list that could obviously benefit from having more mentors and are
looking for volunteers.
> 
> I think Gossip falls into the latter category and would welcome volunteer mentors. So
consider this a call for additional mentors.
> 
> In recent incubator discussions there's been some debate about the role of champion and
how it relates to mentoring. I'm personally not sure where I stand in that debate, and so
have not commented. I accepted the role of champion with the intent of also acting as a mentor
going forward.
> 
> Is that (champion also acting as a mentor) something that we (IPMC) think should be explicitly
called out in the proposal?
> 
> -Taylor
> 
>> On Apr 18, 2016, at 8:35 PM, Henry Saputra <henry.saputra@gmail.com> wrote:
>> 
>> I would recommend getting more mentors to help. Especially almost half of
>> initial committers are not from Apache and the other half seemed very busy
>> with other projects.
>> 
>> - Henry
>> 
>>> On Mon, Apr 18, 2016 at 12:42 PM, P. Taylor Goetz <ptgoetz@gmail.com> wrote:
>>> 
>>> Bump.
>>> 
>>> Are there any further comments/questions on the proposal? Or should we
>>> proceed with a VOTE?
>>> 
>>> -Taylor
>>> 
>>> 
>>>> On Apr 13, 2016, at 6:58 PM, Edward Capriolo <edlinuxguru@gmail.com>
>>> wrote:
>>>> 
>>>> Hello all,
>>>> 
>>>> I would like to open up a discussion on the following proposal:
>>>> https://wiki.apache.org/incubator/GossipProposal
>>>> 
>>>> Besides those  mentioned in the proposal, thank you to Jake Ferrel, Lewis
>>>> McGibbne, and Nate McCall who were a tremendous help!
>>>> 
>>>> A copy of the proposal is below:
>>>> 
>>>> = Abstract =
>>>> 
>>>> Apache Gossip will be an implementation of the Gossip Protocol based
>>>> on code available here: https://github.com/edwardcapriolo/gossip/
>>>> which is already licenced using the glorious Apache V2 License.
>>>> 
>>>> = Proposal =
>>>> 
>>>> Apache Gossip aims to provide a gossip based consensus protocol
>>>> written in Java for peer-to-peer communication to the Apache Incubator
>>>> (http://incubator.apache.org/). This implementation will effectively
>>>> scale from one to one-thousand node clusters. In addition to the code
>>>> implementation, the project should produce specifications of the wire
>>>> protocol, features, and expected behavior of the system such that
>>>> compatible implementations can communicate.
>>>> 
>>>> = Background =
>>>> 
>>>> The gossip protocol has been implemented to varying levels of rigor by
>>>> a number of entities. In particular, Apache Cassandra uses an
>>>> implementation of gossip to locate peers and transmit up/down state.
>>>> Apache Spark leverages tooling in Akka which provides peer-to-peer
>>>> node discovery capabilities.
>>>> 
>>>> *
>>> http://highscalability.com/blog/2011/11/14/using-gossip-protocols-for-failure-detection-monitoring-mess.html
>>>> 
>>>> * https://en.wikipedia.org/wiki/Gossip_protocol
>>>> 
>>>> = Rationale =
>>>> 
>>>> With distributed computing becoming extremely widespread, and the
>>>> growth of the buzz-factor of ‘the-internet-of-things’ it is
>>>> increasingly important that networks of IP addressable devices can
>>>> form a peer-to-peer network. Applications of peer-to-peer networks
>>>> include generating crypto currency, managing hardware such as solar
>>>> power micro-grids, and more traditional roles like grid/High
>>>> Performance Computing and distributed storage systems. Different
>>>> implements of gossip based consensus protocols have been implemented
>>>> in numerous languages or as part of more complex software stacks. The
>>>> Apache foundation should lead the effort of producing a purpose built
>>>> tool that can be used by downstream projects to form peer-to-peer
>>>> networks.
>>>> 
>>>> = Initial Goals =
>>>> 
>>>> * Migration of current code https://github.com/edwardcapriolo/gossip
>>>> to the Apache Software Foundation
>>>> * Secure communications
>>>> * Transport security using a pre-shared key
>>>> * Public Key Infrastructure
>>>> * Introduce a cluster name to wire protocol to avoid misconfigurations
>>>> * Effectively operate when systems have multiple network interfaces
>>>> by controlling IP binding settings
>>>> * Effectively operate when systems have Network Address Translations
>>>> devices between them using a broadcast IP settings
>>>> * Develop advanced integration testing from cluster sizes of 1-1000 nodes
>>>> * Test convergence times
>>>> * Demonstrate the tradeoffs of different settings in regard to
>>>> bandwidth/cpu/convergence time/accuracy
>>>> * Gossip data other than cluster state such as application/user data
>>>> * Provide detailed specifications such that others can implement the
>>>> protocol in other programming languages
>>>> * Explore HTTP transport as an alternative to UDP
>>>> 
>>>> = Current Status =
>>>> 
>>>> The current code has been around for some time. Previously it was a
>>>> google code project. Since the fork in January 2015 there have been 55
>>>> commits and 4 releases.
>>>> 
>>>> == Meritocracy ==
>>>> 
>>>> We believe in meritocracy. All suggestions are taken seriously. We
>>>> enjoy helping new people become part of process. For other projects
>>>> available on our github, once a user shows enough activity we grant
>>>> them collaborator status.
>>>> 
>>>> == Community ==
>>>> 
>>>> In a relatively short amount of time, with a small amount of promotion
>>>> on twitter and through blogging, we have 50+ followers on github and
>>>> several forks of the project.. With the Apache brand we should be able
>>>> to attract more. Once we have entered the incubator we believe it will
>>>> be easier to attempt to unify with other similar implementations.
>>>> 
>>>> == Core Developers ==
>>>> 
>>>> The code was forked on Jan 9th 2015, since then there have been 4
>>>> releases and 55 commits. The majority of th work is by Edward
>>>> Capriolo. Several people are interested in the features of this
>>>> proposal and have indicated they will volunteer their time.
>>>> 
>>>> == Alignment ==
>>>> 
>>>> Apache is the perfect organization to take on this Gossip project.
>>>> Besides benefiting a number of projects directly, the active
>>>> development and reach will help Gossip be adopted and a leader in the
>>>> space.
>>>> 
>>>> = Known Risks =
>>>> 
>>>> Several existing implementations of similar cluster membership systems
>>>> (gossip based and otherwise) exist. A key challenge is moving from a
>>>> relatively niche technical audience to a more general tool for solving
>>>> a common problem. Differentiating when Apache Gossip may be the
>>>> optimal solution versus a clouded landscape of coordination services
>>>> such as etcd or zookeeper, and distributed data stores via feature set
>>>> will also be key. We believe that users be attracted to the
>>>> peer-to-peer distributed service toolkit that gossip will provide.
>>>> 
>>>> == Orphaned products ==
>>>> 
>>>> We plan on building on the current code by developing discrete
>>>> features with a focus on testing. Up until this point the project has
>>>> been maintained be a single person. However the project currently
>>>> releases artifacts to maven central, is tested using travis CI, and
>>>> follows controlled development practice. This level of dedication will
>>>> see the process through the initial stages.
>>>> 
>>>> == Inexperience with Open Source ==
>>>> We are very familiar with Open Source development and the Apache
>>>> Foundation. The current code base already carries an Apache V2
>>>> Licence.
>>>> == Homogenous Developers ==
>>>> Multiple people have made contributions to the current code base. This
>>>> proposal has generated more interest and several more are offering to
>>>> volunteer time. These volunteers are from diverse corporate entities.
>>>> == Reliance on Salaried Developers ==
>>>> We wish to create Apache Gossip for the challenge of producing great
>>>> software. Initially all members of the project will volunteer their
>>>> time and no one will be expressly salaried to work only on this
>>>> project.
>>>> 
>>>> == Relationships with Other Apache Products ==
>>>> If the Apache Gossip project is successful, other products in Apache
>>>> such as Storm or Cassandra could adopt it. However, adoption by those
>>>> specific projects is not our criteria for success. There are a large
>>>> number of applications for this system. One example is Apache JMeter
>>>> could be built with Apache Gossip as a backend for distributed
>>>> testing. Another example is a polyglot registry of thrift services
>>>> with gossip based discovery.
>>>> == A Excessive Fascination with the Apache Brand ==
>>>> We care about the Apache foundation. Having the recognition of the
>>>> Apache incubator will undoubtedly help the project. We do not seek the
>>>> Apache brand to be used as legal shield or personal glory. We believe
>>>> in the Apache foundation and will manage the project with espirit de
>>>> corps, welcoming all through meritocracy while using bylaws as guiding
>>>> values.
>>>> 
>>>> = Initial Source =
>>>> The initial source for https://github.com/edwardcapriolo/gossip was
>>>> forked from https://code.google.com/p/java-gossip/. The code base
>>>> always Apache V2 licensed.
>>>> 
>>>> = Source and Intellectual Property Submission Plan =
>>>> 1. Make final commit on teknek-gossip explaining the move to ASF.
>>>> 1. Move code into ASF repo.
>>>> 1. Rename references to old name.
>>>> 1. Apply Apache V2 licence to all source files.
>>>> 
>>>> = External Dependencies =
>>>> Currently the project encodes messages into JSON for network
>>>> transmission. This is done using JSONorg, but will switch to jackson
>>>> (potentially before the move to the ASF).
>>>> = Cryptography =
>>>> The current code is not using cryptography. It is on the road map to
>>>> add security through transport encryption (SSL) and transport
>>>> encryption.
>>>> = Required Resources =
>>>> == Mailing lists ==
>>>> * dev@gossip.incubator.apache.org
>>>> * commits@gossip.incubator.apache.org
>>>> * private@gossip.incubator.apache.org
>>>> The user list will be added when we have broader adoption.
>>>> == Git Repository ==
>>> https://git-wip-us.apache.org/repos/asf/incubator-gossip.git
>>>> == Issue Tracking ==
>>>> JIRA tracker: GOSSIP
>>>> = Initial Committers =
>>>> * Edward Capriolo (Hive Committer, PMC)
>>>> * P. Taylor Goetz (Storm PMC)
>>>> * Gary Dusbabek (Cassandra Committer, PMC)
>>>> * Dorian Ellerbe (requires CLA)
>>>> * Sathish Dhinakaran (requires CLA)
>>>> = Affiliations =
>>>> With diverse contributors the project will be able to make balanced
>>>> decisions best for the future of the project.
>>>> = Additional Interested Contributors =
>>>> 
>>>> Those interested in getting involved with the project as it starts are
>>>> encourage to list themselves here.
>>>> 
>>>> * (next entry here)
>>>> 
>>>> = Sponsors =
>>>> == Champion ==
>>>> P. Taylor Goetz has offered to champion.
>>>> == Mentors ==
>>>> * Sean Busbey (ASF Member, IPMC, various Apache PMCs, YCSB maintainer)
>>>> * Josh Elser (ASF Member, IPMC, Apache Accumulo PMC, Apache Calcite PMC)
>>>> = Sponsoring Entity =
>>>> The Incubator
>>> 
>>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message