incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mck <...@apache.org>
Subject [PROPOSAL] SkyWalking - proposal for Apache Incubation
Date Wed, 29 Nov 2017 06:58:26 GMT
Dear Apache Incubator Community,

Please accept the following proposal for presentation and discussion:
  https://wiki.apache.org/incubator/SkyWalkingProposal

SkyWalking is a distributed tracing solution that provides automatic
instrumentation, coming from a community of Chinese contributors. This
community has been involved with and part of the Distributed Tracing
workshops held by Adrian Cole (who maintains and develops Zipkin) and
the OpenTracing initiative.

Sheng Wu reached out to me recently asking me to Champion the proposal
because of my involvement with OpenTracing, Zipkin, and these
Distributed Tracing workshops. The whole SkyWalking community has
demonstrated a keenness to join Apache, as is seen on their GitHub
issues discussing the matter. I'm excited to have been asked and have
gladly accepted. Furthermore at last year's ApacheCon in Vancouver I met
Luke Han during the ASF media workshop and watched his presentation
about the challenges of opening up ASF to chinese communities and
developers. Because of this we have reached out to Luke Han as an
additional mentor. The result of this was that Sheng met Luke in person
last weekend in Shanghai. Sheng also met Nicolas Hedhman in Shanghai.
And Willem Ning Jiang has also been added as a mentor, who is also from
Huawei and is currently involved in the ServiceComb proposal. Otherwise
I'm aware that I'm new to the Incubator and its processes, so any
additional mentors familiar with the finer details and precedence will
be most welcomed.

regards,
Mick

----

= Abstract =
Skywalking is an APM (application performance monitor), especially for
microservice, Cloud Native and container-based architecture systems.
Also known as a distributed tracing system. It provides an automatic way
to instrument applications: no need to change any of the source code of
the target application; and an collector with an very high efficiency
streaming module.

= Proposal =
The goal of this proposal is to bring the existing Skywalking
https://github.com/OpenSkywalking/skywalking codebase and existing
developers and community into the Apache Software Foundation (ASF) in
order to build a global, diverse and self-governed open source community
in APM field. 

This project started in Open Source on GitHub about more than 2 years
ago. Beginning with a small SDK and collector. So far the OpenSkywalking
https://github.com/OpenSkywalking/Organization governs the project
through the PMC and Committer Team. The major contributors are from
Huawei DevCloud Team, Tydic, Oneapm (APM vendor),  Alibaba Group,
dangdang.com and cloudwise (APM vendor). 

OpenSkywalking is submitting this proposal to donate the Skywalking
sources code and  associated artifacts (documentation, web site content,
wiki, etc.) to the Apache Software Foundation Incubator under the Apache
License, Version 2.0. These artifacts are currently available on GitHub
at https://github.com/OpenSkywalking and include:
 * Skywalking: The java sniffer(agent) for collecting data, and
 collector for analysing and persistence.
 * Skywalking-UI: The web UI for skywalking APM

Voted on submitting the proposal to the Incubator.
https://github.com/OpenSkywalking/Organization/issues/11

= Background =
Mircro-service, Cloud Native and container-based architecture system are
becoming more and more popular, so the traditional monitoring, like
application loggings, can provide less information because of the
distributed isolates the relations. Based on the Google Dapper paper
https://research.google.com/pubs/pub36356.html, many tracing systems
born. The OpenSkywalking organisation was created with  Skywalking made
based on tracing, but not just tracing, it adds additional value by
reducing the sniffer (agent) cost, analysis and visualization. 

In 2015, Skywalking project started, when service-oriented architecture
became popular. At first, skywalking provided a very simple SDK, and
collected data into a HBASE cluster. After we opened on the GitHub, the
community gives the feedbacks about how difficult to maintain a HBase
cluster, even harder than the applications under monitored. So, in 2.x
2016, skywalking provided a self-designed storage, and update the SDK to
a javaagent with supporting auto-instrumentation. Then since 2017, more
and more contributors joined, we set up the PMC team and committer team.
Skywalking evolved to an APM, and more and more features provided since
then.

= Rationale =
Skywalking includes these primary parts:
 1. Provide an anto-instrument sniffer, which is based on Javaagent and
 collects events and traces happened inside JVM, with little CPU/Memory
 cost.
 2. An extendable `tracing data protocol suit` with gRPC and HTTP
 implementations, is compatible for other language agent or SDK. 
 3. Provide Collector, which accepts the `tracing data protocol suit`,
 and does the analysis and aggregation inside for detecting the
 relationships among applications and services, generating the metrics,
 and altering.
 4. Provided our own UI, which visualizes the topological graph of
 related applications and services, trace stack, metrics and alerting.

Also, Skywalking team is passionate about community cooperations.
Skywalking is a supported tracer and member of OpenTracing
http://opentracing.io . Also we take part in the  TraceContext Specs
https://github.com/TraceContext/tracecontext-spec , which is about
`tracing context propagation format`. The founder of the project, Sheng
Wu, is the member of these organizations, 

There is a strong need for an open, easy-to-use APM towards helping
today's DevOps people to monitor their running systems, while also being
easy and transparent to maintain and modify.
The Skywalking has been an hot open source project in GitHub. We feel
that by moving to Apache it will help us work in a more global and open
way, presenting the project as a commercial friendly but vendor neutral
technology. Under Apache's strong governance and existing processes,
hope is also to make the Skywalking releases more reliable, as is
crucial for monitoring systems running in any Application Zone.

= Initial Goals =
Our initial goals are to bring Skywalking into the ASF. The most
important things are following ASF's governance module, and integrating
with Apache development process. Also moving the existing codebase to an
Apache git repository.

= Current Status =
== Meritocracy ==
Skywalking was originally created by Sheng Wu,  吴晟
https://github.com/wu-sheng himself in May 2015, and opened in GitHub in
Nov. 8th, 2015. The project now has committers and users from many
companies. The newer committers of the project are guided by the existed
PMC and Committer Team members, by reviewing and discussing with them.
When they are ready, PMC will start a vote to promote him/her to become
a member of PMC and Committer Team.
Also after the founder, Sheng Wu, joined Huawei in 2017, Huawei DevCloud
decided to contribute and support the project. So he personally leads
the interested developers, and helps them to join the community quickly
and efficiently. Now we have set `Huawei Devcloud Contributor Team` for
them.
Contributions are always welcomed, highly valued. A lot of energy is
allocated to ensure help to all wishing to contribute.

== Community ==
OpenSkwaylking is supported by 5 companies: Huawei, shurenyun.com,
dangdang.com, tydic, nutz.cn , which are both contributing to the
project and also used it in product, also provide many important
feedbacks from their actual scenarios.
The contributors, who can provide employer informantions, are from
Huawei DevCloud Team, Tydic, Oneapm (APM vendor), Alibaba Group,
dangdang.com, cloudwise (APM vendor), jd.com, cig.cn, hecom.cn, and
www.suixingpay.com.

== Core Developers ==
The core developers are a diverse group of experienced open source
developers and team leaders.
 * PMC members –
 https://github.com/OpenSkywalking/Organization/blob/master/README.md#project-management-committee-pmc
 * Committer Team members –
 https://github.com/OpenSkywalking/Organization/blob/master/README.md#committer-team

== Known Risks ==
The project is well known in global tracing and APM field, and live more
than 2 years. We are not sure there exists a risk, but definitely have a
challenge for us. So far, the contributors, committers and confirmed
product use cases are all from China. 
=== Language and Culture ===
Concerns have been raised about language challenges and, as is typical
for developers not strong in english, aware that the community can be
shy in a debate or disagreement when english is used. Having looked
through the project's history this concern can be seen to be minor. The
commitlog is in english, and so are the tickets and the pull requests.
The website and documentation is equally both english and chinese. The
community has been strict upon itself to ensure it became and stays so.
The use of chinese should not be seen as a problem, it is a multilingual
world, but it is important that english speakers feel included and able
to contribute freely to the project without having to ask for
translations. To deal with poor or broken english it's important that a
gentler and inclusive community is fostered.

== Orphaned products ==
The contributors and community cooperation brings the project from a
small concept into a real APM system. So far, many companies, e.g.
Huawei DevCloud, dangdang.com, tydic, jd.com, have their own
independency contribute team for Skywalking. So this is not risk of any
signs of orphaned or abandoned code.

== Inexperience with Open Source ==
The founder of the project, Sheng Wu, is an open source lover. He has
contributed for these open source projects: OpenTracing-Specification,
OpenTracing-Java, Motan, Hprose, OpenTracing specification-zh,
apache/incubator-rocketmq.

Hongtao Gao, our committer team member, is the major maintainer of 
sharding-jdbc https://github.com/shardingjdbc/sharding-jdbc and
elastic-job https://github.com/elasticjob/elastic-job since 2014.

The project are following these two experienced open source people's
guidance. And glad to learn from the Apache Way.

== Homogenous Developers ==
The confirmed contributors list:
https://github.com/OpenSkywalking/Organization/blob/master/README.md#contributors.
As mentioned above in the Community section.

== Reliance on Salaried Developers ==
First of all, most members of the PMC and Committer Team, are paid by
their employers (Huawei, Tydic, dangdang.com and OneAPM) to contribute
to this project. But we have to say, many of them have more than one
employer in the past 3 years contribution, but the contributions are
never stopped. We don't just depend on salaried contributors only.
Skywalking itself and the distributed tracing field are very attractive
and important for every company the contributors work in, and they have
no reason to stop them. Contributions and new committers are expected.

== Relationships with Other Apache Products ==
HBase was used as a backend before being rewritten, and that ZooKeeper,
httpcomponents, log4j and junit are used. Also we are considering all
java projects, especially big data technology, e.g. Hadoop, Hbase,
Cassandra.

== A Excessive Fascination with the Apache Brand ==
The ASF has a strong brand, and that brand is in itself attractive.
Skywalking will benefit from world wide collaboration and use cases. The
Apache brand will be duly respected and honored in regards to  marketing
and publicity. Further more, with the help of Incubator PMC, we will
make every effort to follow the rules and policies of the ASF.

== Documentation ==
 * https://github.com/OpenSkywalking/skywalking/tree/master/docs
 * https://github.com/OpenSkywalking/skywalking/wiki

== Initial Source ==
The project consists of two distinct codebases: the server and the UI.
These have existed as separate git repositories.
 * https://github.com/OpenSkywalking/skywalking
 * https://github.com/OpenSkywalking/skywalking-ui

== Source and Intellectual Property Submission Plan ==
The project is under Apache Licensed 2.0 already. As soon as Skywalking
is approved to join Apache, we can transfer of source code to the Apache
Foundation. There is no legal issue.
All source artifacts (code and documentation) will be assigned copyright
to the ASF. Currently all artifacts are copyright to 'OpenSkywalking
Organization', so effort in this process pertains mostly to obtaining
the individual and corporation contributor agreements (ICLAs and CCLAs)
from all committers and PMC.

== External Dependencies ==
 * Byte-buddy (Apache 2.0)
 * gRPC (Apache 2.0)
 * netty (Apache 2.0)
 * guava (Apache 2.0)
 * elasticsearch (Apache 2.0)
 * H2 Driver (MPL 2.0 or EPL 1.0)
 * Zookeeper Client (Apache 2.0)
 * disruptor (Apache 2.0)
 * junit (MPL)
 * mockito (MIT)
 * powermock (Apache 2.0)
 * Spring (Apache 2.0)
 * gson  (Apache 2.0)
 * httpcomponents (Apache 2.0)
 * log4j  (Apache 2.0)
 * vis (Apache 2.0)
 * webjars-boostrap  (Apache 2.0)
 * animate.css (MIT)
 * jquery (js.foundation)
 * jquery-ui (js.foundation)
 * jsView (MIT)
 * metisMenu (MIT)
 * d3 (BSD-3)
 * echarts (BSD-3)
 * DataTables (MIT)
 * requirejs (MIT)
 * vue (MIT)

= Required Resources =
== Mailing Lists ==
 * private@skywalking.incubator.apache.org (moderated subscriptions)
 * commits@skywalking.incubator.apache.org
 * dev@skywalking.incubator.apache.org
 * user@skywalking.incubator.apache.org

== Git Repositories ==
 * https://git-wip-us.apache.org/repos/asf/incubator-skywalking.git
 * https://git-wip-us.apache.org/repos/asf/incubator-skywalking-ui.git

== Issue Tracking ==
GitHub Issues or JIRA Project Skywalking.
To be discussed further, there's a desire to continue to use GitHub
issues and it's now visible that other projects are using this. 

== Initial Incubator PMC ==
 * 吴晟, Sheng Wu  @wu-sheng
 * 彭勇升, Yongsheng Peng  @peng-yongsheng
 * 张鑫, Xin Zhang  @ascrutae

== Initial Committer ==
 * 吴晟, Sheng Wu  @wu-sheng
 * 彭勇升, Yongsheng Peng  @peng-yongsheng
 * 张鑫, Xin Zhang  @ascrutae
 * 高洪涛, Hongtao Gao  @hanahmily
 * 柏杨, Yang Bai  @bai-yang
 * 王凯, Kai Wang  @oracle219
 * 李运涛, Yuntao Li  @lytscu
 * 汪盛, Sheng Wang  @titsquid
 * 司冬雪, Dongxue Si  @IluckySi
 * 张科伟, Kewei Zhang  @zhangkewei

= Champion =
 * Michael Semb Wever, mck@apache.org

Voted for skywalking Incubator champion.
https://github.com/OpenSkywalking/Organization/issues/12

= Mentors =
 * Michael Semb Wever, mck@apache.org
 * Luke Han, lukehan@apache.org
 * Willem Ning Jiang, ningjiang@apache.org

Skywalking received a lot helps from Michael Semb Wever about this
proposal and Apache way. Set face-to-face meeting with three Apache
members: Luke Han, Nicolas Hedhman, Willem Ning Jiang. They helps us a
lot.

== Sponsoring Entity ==
We are requesting the Incubator to sponsor this project.

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message