incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Saputra <henry.sapu...@gmail.com>
Subject Re: [PROPOSAL] Gossip incubator proposal
Date Tue, 19 Apr 2016 00:35:01 GMT
I would recommend getting more mentors to help. Especially almost half of
initial committers are not from Apache and the other half seemed very busy
with other projects.

- Henry

On Mon, Apr 18, 2016 at 12:42 PM, P. Taylor Goetz <ptgoetz@gmail.com> wrote:

> Bump.
>
> Are there any further comments/questions on the proposal? Or should we
> proceed with a VOTE?
>
> -Taylor
>
>
> > On Apr 13, 2016, at 6:58 PM, Edward Capriolo <edlinuxguru@gmail.com>
> wrote:
> >
> > Hello all,
> >
> > I would like to open up a discussion on the following proposal:
> > https://wiki.apache.org/incubator/GossipProposal
> >
> > Besides those  mentioned in the proposal, thank you to Jake Ferrel, Lewis
> > McGibbne, and Nate McCall who were a tremendous help!
> >
> > A copy of the proposal is below:
> >
> > = Abstract =
> >
> > Apache Gossip will be an implementation of the Gossip Protocol based
> > on code available here: https://github.com/edwardcapriolo/gossip/
> > which is already licenced using the glorious Apache V2 License.
> >
> > = Proposal =
> >
> > Apache Gossip aims to provide a gossip based consensus protocol
> > written in Java for peer-to-peer communication to the Apache Incubator
> > (http://incubator.apache.org/). This implementation will effectively
> > scale from one to one-thousand node clusters. In addition to the code
> > implementation, the project should produce specifications of the wire
> > protocol, features, and expected behavior of the system such that
> > compatible implementations can communicate.
> >
> > = Background =
> >
> > The gossip protocol has been implemented to varying levels of rigor by
> > a number of entities. In particular, Apache Cassandra uses an
> > implementation of gossip to locate peers and transmit up/down state.
> > Apache Spark leverages tooling in Akka which provides peer-to-peer
> > node discovery capabilities.
> >
> > *
> http://highscalability.com/blog/2011/11/14/using-gossip-protocols-for-failure-detection-monitoring-mess.html
> >
> > * https://en.wikipedia.org/wiki/Gossip_protocol
> >
> > = Rationale =
> >
> > With distributed computing becoming extremely widespread, and the
> > growth of the buzz-factor of ‘the-internet-of-things’ it is
> > increasingly important that networks of IP addressable devices can
> > form a peer-to-peer network. Applications of peer-to-peer networks
> > include generating crypto currency, managing hardware such as solar
> > power micro-grids, and more traditional roles like grid/High
> > Performance Computing and distributed storage systems. Different
> > implements of gossip based consensus protocols have been implemented
> > in numerous languages or as part of more complex software stacks. The
> > Apache foundation should lead the effort of producing a purpose built
> > tool that can be used by downstream projects to form peer-to-peer
> > networks.
> >
> > = Initial Goals =
> >
> > * Migration of current code https://github.com/edwardcapriolo/gossip
> > to the Apache Software Foundation
> > * Secure communications
> >  * Transport security using a pre-shared key
> >  * Public Key Infrastructure
> > * Introduce a cluster name to wire protocol to avoid misconfigurations
> > * Effectively operate when systems have multiple network interfaces
> > by controlling IP binding settings
> > * Effectively operate when systems have Network Address Translations
> > devices between them using a broadcast IP settings
> > * Develop advanced integration testing from cluster sizes of 1-1000 nodes
> >  * Test convergence times
> >  * Demonstrate the tradeoffs of different settings in regard to
> > bandwidth/cpu/convergence time/accuracy
> > * Gossip data other than cluster state such as application/user data
> > * Provide detailed specifications such that others can implement the
> > protocol in other programming languages
> > * Explore HTTP transport as an alternative to UDP
> >
> > = Current Status =
> >
> > The current code has been around for some time. Previously it was a
> > google code project. Since the fork in January 2015 there have been 55
> > commits and 4 releases.
> >
> > == Meritocracy ==
> >
> > We believe in meritocracy. All suggestions are taken seriously. We
> > enjoy helping new people become part of process. For other projects
> > available on our github, once a user shows enough activity we grant
> > them collaborator status.
> >
> > == Community ==
> >
> > In a relatively short amount of time, with a small amount of promotion
> > on twitter and through blogging, we have 50+ followers on github and
> > several forks of the project.. With the Apache brand we should be able
> > to attract more. Once we have entered the incubator we believe it will
> > be easier to attempt to unify with other similar implementations.
> >
> > == Core Developers ==
> >
> > The code was forked on Jan 9th 2015, since then there have been 4
> > releases and 55 commits. The majority of th work is by Edward
> > Capriolo. Several people are interested in the features of this
> > proposal and have indicated they will volunteer their time.
> >
> > == Alignment ==
> >
> > Apache is the perfect organization to take on this Gossip project.
> > Besides benefiting a number of projects directly, the active
> > development and reach will help Gossip be adopted and a leader in the
> > space.
> >
> > = Known Risks =
> >
> > Several existing implementations of similar cluster membership systems
> > (gossip based and otherwise) exist. A key challenge is moving from a
> > relatively niche technical audience to a more general tool for solving
> > a common problem. Differentiating when Apache Gossip may be the
> > optimal solution versus a clouded landscape of coordination services
> > such as etcd or zookeeper, and distributed data stores via feature set
> > will also be key. We believe that users be attracted to the
> > peer-to-peer distributed service toolkit that gossip will provide.
> >
> > == Orphaned products ==
> >
> > We plan on building on the current code by developing discrete
> > features with a focus on testing. Up until this point the project has
> > been maintained be a single person. However the project currently
> > releases artifacts to maven central, is tested using travis CI, and
> > follows controlled development practice. This level of dedication will
> > see the process through the initial stages.
> >
> > == Inexperience with Open Source ==
> > We are very familiar with Open Source development and the Apache
> > Foundation. The current code base already carries an Apache V2
> > Licence.
> > == Homogenous Developers ==
> > Multiple people have made contributions to the current code base. This
> > proposal has generated more interest and several more are offering to
> > volunteer time. These volunteers are from diverse corporate entities.
> > == Reliance on Salaried Developers ==
> > We wish to create Apache Gossip for the challenge of producing great
> > software. Initially all members of the project will volunteer their
> > time and no one will be expressly salaried to work only on this
> > project.
> >
> > == Relationships with Other Apache Products ==
> > If the Apache Gossip project is successful, other products in Apache
> > such as Storm or Cassandra could adopt it. However, adoption by those
> > specific projects is not our criteria for success. There are a large
> > number of applications for this system. One example is Apache JMeter
> > could be built with Apache Gossip as a backend for distributed
> > testing. Another example is a polyglot registry of thrift services
> > with gossip based discovery.
> > == A Excessive Fascination with the Apache Brand ==
> > We care about the Apache foundation. Having the recognition of the
> > Apache incubator will undoubtedly help the project. We do not seek the
> > Apache brand to be used as legal shield or personal glory. We believe
> > in the Apache foundation and will manage the project with espirit de
> > corps, welcoming all through meritocracy while using bylaws as guiding
> > values.
> >
> > = Initial Source =
> > The initial source for https://github.com/edwardcapriolo/gossip was
> > forked from https://code.google.com/p/java-gossip/. The code base
> > always Apache V2 licensed.
> >
> > = Source and Intellectual Property Submission Plan =
> > 1. Make final commit on teknek-gossip explaining the move to ASF.
> > 1. Move code into ASF repo.
> > 1. Rename references to old name.
> > 1. Apply Apache V2 licence to all source files.
> >
> > = External Dependencies =
> > Currently the project encodes messages into JSON for network
> > transmission. This is done using JSONorg, but will switch to jackson
> > (potentially before the move to the ASF).
> > = Cryptography =
> > The current code is not using cryptography. It is on the road map to
> > add security through transport encryption (SSL) and transport
> > encryption.
> > = Required Resources =
> > == Mailing lists ==
> > * dev@gossip.incubator.apache.org
> > * commits@gossip.incubator.apache.org
> > * private@gossip.incubator.apache.org
> > The user list will be added when we have broader adoption.
> > == Git Repository ==
> https://git-wip-us.apache.org/repos/asf/incubator-gossip.git
> > == Issue Tracking ==
> > JIRA tracker: GOSSIP
> > = Initial Committers =
> > * Edward Capriolo (Hive Committer, PMC)
> > * P. Taylor Goetz (Storm PMC)
> > * Gary Dusbabek (Cassandra Committer, PMC)
> > * Dorian Ellerbe (requires CLA)
> > * Sathish Dhinakaran (requires CLA)
> > = Affiliations =
> > With diverse contributors the project will be able to make balanced
> > decisions best for the future of the project.
> > = Additional Interested Contributors =
> >
> > Those interested in getting involved with the project as it starts are
> > encourage to list themselves here.
> >
> > * (next entry here)
> >
> > = Sponsors =
> > == Champion ==
> > P. Taylor Goetz has offered to champion.
> > == Mentors ==
> > * Sean Busbey (ASF Member, IPMC, various Apache PMCs, YCSB maintainer)
> > * Josh Elser (ASF Member, IPMC, Apache Accumulo PMC, Apache Calcite PMC)
> > = Sponsoring Entity =
> > The Incubator
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message