incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "P. Taylor Goetz" <ptgo...@gmail.com>
Subject Re: [PROPOSAL] Gossip incubator proposal
Date Tue, 19 Apr 2016 01:28:46 GMT
Thanks Julian,

I updated the proposal to be more explicit regarding my intentions to act as a mentor as well.

-Taylor
> On Apr 18, 2016, at 8:59 PM, Julian Hyde <jhyde@apache.org> wrote:
> 
> In the Quickstep proposal, Roman explicitly listed himself as both Champion and Mentor.
I think that is a good model to follow. If someone lists themselves only as Champion would
I assume that their responsibilities end shortly after incubation starts.
> 
> To use the “it takes a village” metaphor, I think that incubator projects really
need one or two committed parents rather than lots and lots of part-time aunts and uncles.
> 
> Julian
> 
> 
>> On Apr 18, 2016, at 5:52 PM, P. Taylor Goetz <ptgoetz@gmail.com> wrote:
>> 
>> Thanks for the input Henry!
>> 
>> I've seen proposals in the past come with what seems like a fully-baked list of mentors,
and others with a short list that could obviously benefit from having more mentors and are
looking for volunteers.
>> 
>> I think Gossip falls into the latter category and would welcome volunteer mentors.
So consider this a call for additional mentors.
>> 
>> In recent incubator discussions there's been some debate about the role of champion
and how it relates to mentoring. I'm personally not sure where I stand in that debate, and
so have not commented. I accepted the role of champion with the intent of also acting as a
mentor going forward.
>> 
>> Is that (champion also acting as a mentor) something that we (IPMC) think should
be explicitly called out in the proposal?
>> 
>> -Taylor
>> 
>>> On Apr 18, 2016, at 8:35 PM, Henry Saputra <henry.saputra@gmail.com> wrote:
>>> 
>>> I would recommend getting more mentors to help. Especially almost half of
>>> initial committers are not from Apache and the other half seemed very busy
>>> with other projects.
>>> 
>>> - Henry
>>> 
>>>> On Mon, Apr 18, 2016 at 12:42 PM, P. Taylor Goetz <ptgoetz@gmail.com>
wrote:
>>>> 
>>>> Bump.
>>>> 
>>>> Are there any further comments/questions on the proposal? Or should we
>>>> proceed with a VOTE?
>>>> 
>>>> -Taylor
>>>> 
>>>> 
>>>>> On Apr 13, 2016, at 6:58 PM, Edward Capriolo <edlinuxguru@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hello all,
>>>>> 
>>>>> I would like to open up a discussion on the following proposal:
>>>>> https://wiki.apache.org/incubator/GossipProposal
>>>>> 
>>>>> Besides those  mentioned in the proposal, thank you to Jake Ferrel, Lewis
>>>>> McGibbne, and Nate McCall who were a tremendous help!
>>>>> 
>>>>> A copy of the proposal is below:
>>>>> 
>>>>> = Abstract =
>>>>> 
>>>>> Apache Gossip will be an implementation of the Gossip Protocol based
>>>>> on code available here: https://github.com/edwardcapriolo/gossip/
>>>>> which is already licenced using the glorious Apache V2 License.
>>>>> 
>>>>> = Proposal =
>>>>> 
>>>>> Apache Gossip aims to provide a gossip based consensus protocol
>>>>> written in Java for peer-to-peer communication to the Apache Incubator
>>>>> (http://incubator.apache.org/). This implementation will effectively
>>>>> scale from one to one-thousand node clusters. In addition to the code
>>>>> implementation, the project should produce specifications of the wire
>>>>> protocol, features, and expected behavior of the system such that
>>>>> compatible implementations can communicate.
>>>>> 
>>>>> = Background =
>>>>> 
>>>>> The gossip protocol has been implemented to varying levels of rigor by
>>>>> a number of entities. In particular, Apache Cassandra uses an
>>>>> implementation of gossip to locate peers and transmit up/down state.
>>>>> Apache Spark leverages tooling in Akka which provides peer-to-peer
>>>>> node discovery capabilities.
>>>>> 
>>>>> *
>>>> http://highscalability.com/blog/2011/11/14/using-gossip-protocols-for-failure-detection-monitoring-mess.html
>>>>> 
>>>>> * https://en.wikipedia.org/wiki/Gossip_protocol
>>>>> 
>>>>> = Rationale =
>>>>> 
>>>>> With distributed computing becoming extremely widespread, and the
>>>>> growth of the buzz-factor of ‘the-internet-of-things’ it is
>>>>> increasingly important that networks of IP addressable devices can
>>>>> form a peer-to-peer network. Applications of peer-to-peer networks
>>>>> include generating crypto currency, managing hardware such as solar
>>>>> power micro-grids, and more traditional roles like grid/High
>>>>> Performance Computing and distributed storage systems. Different
>>>>> implements of gossip based consensus protocols have been implemented
>>>>> in numerous languages or as part of more complex software stacks. The
>>>>> Apache foundation should lead the effort of producing a purpose built
>>>>> tool that can be used by downstream projects to form peer-to-peer
>>>>> networks.
>>>>> 
>>>>> = Initial Goals =
>>>>> 
>>>>> * Migration of current code https://github.com/edwardcapriolo/gossip
>>>>> to the Apache Software Foundation
>>>>> * Secure communications
>>>>> * Transport security using a pre-shared key
>>>>> * Public Key Infrastructure
>>>>> * Introduce a cluster name to wire protocol to avoid misconfigurations
>>>>> * Effectively operate when systems have multiple network interfaces
>>>>> by controlling IP binding settings
>>>>> * Effectively operate when systems have Network Address Translations
>>>>> devices between them using a broadcast IP settings
>>>>> * Develop advanced integration testing from cluster sizes of 1-1000 nodes
>>>>> * Test convergence times
>>>>> * Demonstrate the tradeoffs of different settings in regard to
>>>>> bandwidth/cpu/convergence time/accuracy
>>>>> * Gossip data other than cluster state such as application/user data
>>>>> * Provide detailed specifications such that others can implement the
>>>>> protocol in other programming languages
>>>>> * Explore HTTP transport as an alternative to UDP
>>>>> 
>>>>> = Current Status =
>>>>> 
>>>>> The current code has been around for some time. Previously it was a
>>>>> google code project. Since the fork in January 2015 there have been 55
>>>>> commits and 4 releases.
>>>>> 
>>>>> == Meritocracy ==
>>>>> 
>>>>> We believe in meritocracy. All suggestions are taken seriously. We
>>>>> enjoy helping new people become part of process. For other projects
>>>>> available on our github, once a user shows enough activity we grant
>>>>> them collaborator status.
>>>>> 
>>>>> == Community ==
>>>>> 
>>>>> In a relatively short amount of time, with a small amount of promotion
>>>>> on twitter and through blogging, we have 50+ followers on github and
>>>>> several forks of the project.. With the Apache brand we should be able
>>>>> to attract more. Once we have entered the incubator we believe it will
>>>>> be easier to attempt to unify with other similar implementations.
>>>>> 
>>>>> == Core Developers ==
>>>>> 
>>>>> The code was forked on Jan 9th 2015, since then there have been 4
>>>>> releases and 55 commits. The majority of th work is by Edward
>>>>> Capriolo. Several people are interested in the features of this
>>>>> proposal and have indicated they will volunteer their time.
>>>>> 
>>>>> == Alignment ==
>>>>> 
>>>>> Apache is the perfect organization to take on this Gossip project.
>>>>> Besides benefiting a number of projects directly, the active
>>>>> development and reach will help Gossip be adopted and a leader in the
>>>>> space.
>>>>> 
>>>>> = Known Risks =
>>>>> 
>>>>> Several existing implementations of similar cluster membership systems
>>>>> (gossip based and otherwise) exist. A key challenge is moving from a
>>>>> relatively niche technical audience to a more general tool for solving
>>>>> a common problem. Differentiating when Apache Gossip may be the
>>>>> optimal solution versus a clouded landscape of coordination services
>>>>> such as etcd or zookeeper, and distributed data stores via feature set
>>>>> will also be key. We believe that users be attracted to the
>>>>> peer-to-peer distributed service toolkit that gossip will provide.
>>>>> 
>>>>> == Orphaned products ==
>>>>> 
>>>>> We plan on building on the current code by developing discrete
>>>>> features with a focus on testing. Up until this point the project has
>>>>> been maintained be a single person. However the project currently
>>>>> releases artifacts to maven central, is tested using travis CI, and
>>>>> follows controlled development practice. This level of dedication will
>>>>> see the process through the initial stages.
>>>>> 
>>>>> == Inexperience with Open Source ==
>>>>> We are very familiar with Open Source development and the Apache
>>>>> Foundation. The current code base already carries an Apache V2
>>>>> Licence.
>>>>> == Homogenous Developers ==
>>>>> Multiple people have made contributions to the current code base. This
>>>>> proposal has generated more interest and several more are offering to
>>>>> volunteer time. These volunteers are from diverse corporate entities.
>>>>> == Reliance on Salaried Developers ==
>>>>> We wish to create Apache Gossip for the challenge of producing great
>>>>> software. Initially all members of the project will volunteer their
>>>>> time and no one will be expressly salaried to work only on this
>>>>> project.
>>>>> 
>>>>> == Relationships with Other Apache Products ==
>>>>> If the Apache Gossip project is successful, other products in Apache
>>>>> such as Storm or Cassandra could adopt it. However, adoption by those
>>>>> specific projects is not our criteria for success. There are a large
>>>>> number of applications for this system. One example is Apache JMeter
>>>>> could be built with Apache Gossip as a backend for distributed
>>>>> testing. Another example is a polyglot registry of thrift services
>>>>> with gossip based discovery.
>>>>> == A Excessive Fascination with the Apache Brand ==
>>>>> We care about the Apache foundation. Having the recognition of the
>>>>> Apache incubator will undoubtedly help the project. We do not seek the
>>>>> Apache brand to be used as legal shield or personal glory. We believe
>>>>> in the Apache foundation and will manage the project with espirit de
>>>>> corps, welcoming all through meritocracy while using bylaws as guiding
>>>>> values.
>>>>> 
>>>>> = Initial Source =
>>>>> The initial source for https://github.com/edwardcapriolo/gossip was
>>>>> forked from https://code.google.com/p/java-gossip/. The code base
>>>>> always Apache V2 licensed.
>>>>> 
>>>>> = Source and Intellectual Property Submission Plan =
>>>>> 1. Make final commit on teknek-gossip explaining the move to ASF.
>>>>> 1. Move code into ASF repo.
>>>>> 1. Rename references to old name.
>>>>> 1. Apply Apache V2 licence to all source files.
>>>>> 
>>>>> = External Dependencies =
>>>>> Currently the project encodes messages into JSON for network
>>>>> transmission. This is done using JSONorg, but will switch to jackson
>>>>> (potentially before the move to the ASF).
>>>>> = Cryptography =
>>>>> The current code is not using cryptography. It is on the road map to
>>>>> add security through transport encryption (SSL) and transport
>>>>> encryption.
>>>>> = Required Resources =
>>>>> == Mailing lists ==
>>>>> * dev@gossip.incubator.apache.org
>>>>> * commits@gossip.incubator.apache.org
>>>>> * private@gossip.incubator.apache.org
>>>>> The user list will be added when we have broader adoption.
>>>>> == Git Repository ==
>>>> https://git-wip-us.apache.org/repos/asf/incubator-gossip.git
>>>>> == Issue Tracking ==
>>>>> JIRA tracker: GOSSIP
>>>>> = Initial Committers =
>>>>> * Edward Capriolo (Hive Committer, PMC)
>>>>> * P. Taylor Goetz (Storm PMC)
>>>>> * Gary Dusbabek (Cassandra Committer, PMC)
>>>>> * Dorian Ellerbe (requires CLA)
>>>>> * Sathish Dhinakaran (requires CLA)
>>>>> = Affiliations =
>>>>> With diverse contributors the project will be able to make balanced
>>>>> decisions best for the future of the project.
>>>>> = Additional Interested Contributors =
>>>>> 
>>>>> Those interested in getting involved with the project as it starts are
>>>>> encourage to list themselves here.
>>>>> 
>>>>> * (next entry here)
>>>>> 
>>>>> = Sponsors =
>>>>> == Champion ==
>>>>> P. Taylor Goetz has offered to champion.
>>>>> == Mentors ==
>>>>> * Sean Busbey (ASF Member, IPMC, various Apache PMCs, YCSB maintainer)
>>>>> * Josh Elser (ASF Member, IPMC, Apache Accumulo PMC, Apache Calcite PMC)
>>>>> = Sponsoring Entity =
>>>>> The Incubator
>>>> 
>>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message