Return-Path: X-Original-To: apmail-incubator-general-archive@www.apache.org Delivered-To: apmail-incubator-general-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E184518754 for ; Thu, 12 Nov 2015 01:47:39 +0000 (UTC) Received: (qmail 77374 invoked by uid 500); 12 Nov 2015 01:47:39 -0000 Delivered-To: apmail-incubator-general-archive@incubator.apache.org Received: (qmail 77189 invoked by uid 500); 12 Nov 2015 01:47:39 -0000 Mailing-List: contact general-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@incubator.apache.org Delivered-To: mailing list general@incubator.apache.org Received: (qmail 77172 invoked by uid 99); 12 Nov 2015 01:47:39 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Nov 2015 01:47:39 +0000 Received: from mail-ig0-f181.google.com (mail-ig0-f181.google.com [209.85.213.181]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id E4FFF1A0181 for ; Thu, 12 Nov 2015 01:47:38 +0000 (UTC) Received: by igl9 with SMTP id 9so88139112igl.0 for ; Wed, 11 Nov 2015 17:47:37 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.50.62.104 with SMTP id x8mr12068081igr.22.1447292857945; Wed, 11 Nov 2015 17:47:37 -0800 (PST) Received: by 10.36.3.144 with HTTP; Wed, 11 Nov 2015 17:47:37 -0800 (PST) In-Reply-To: References: Date: Wed, 11 Nov 2015 17:47:37 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [DISCUSS] S2Graph Incubator Proposal From: Hyunsik Choi To: "general@incubator.apache.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Luke, Thank you for your interest in S2Graph project. If you don't mind, we'd like to add you to the initial committer list. I think that your experience and skills about HBase would be very helpful to S2Graph project. Best regards, Hyunsik On Mon, Nov 9, 2015 at 6:58 PM, Luke Han wrote: > I'm very interesting about this project, would love to help but I'm not > IPMC member. > > Please let me know if there's anything I could help on. > > Thanks. > > > Best Regards! > --------------------- > > Luke Han > > On Tue, Nov 10, 2015 at 9:03 AM, Hyunsik Choi wrote: > >> Hi Seetharam, >> >> Thank you for your volunteering! I've added your name to the mentor list= . >> >> I also updated the initial committer list and affiliations via google >> search. >> If I wrote wrong affiliations, please let me know. >> >> Best regards, >> Hyunsik >> >> >> On Mon, Nov 9, 2015 at 4:21 PM, Seetharam Venkatesh >> wrote: >> > Hi Hyunsik, >> > >> > If you are still looking for mentors, let me volunteer as one. >> > >> > Thanks! >> > >> > On Mon, Nov 9, 2015 at 3:45 PM Hyunsik Choi wrote= : >> > >> >> Thank you all guys I just put you names on the nominated mentor list= . >> >> >> >> @Andrew, >> >> >> >> I agree with you. S2Graph already has good relationships with other >> >> ASF projects, such as HBase and Spark, In addition, they have a plan >> >> to expand its relationship to Apache incubator TinkerPop, which is a >> >> graph computing framework. I'm looking forward to their combinations. >> >> >> >> @Sergio, >> >> >> >> Thank you for attending the talk and joining the S2Graph mentors. Tha= t >> >> was Doyung Yoon, one of the S2Graph creators. He had a talk at the >> >> last ApacheCon. >> >> >> >> On Mon, Nov 9, 2015 at 11:58 AM, Sergio Fern=C3=A1ndez >> >> wrote: >> >> > Hi Hyunsik, I attended your talk at the last ApacheCon, and I think= S2 >> >> has >> >> > quite some potential. So if you need a mentor, count me in! >> >> > >> >> > On Mon, Nov 9, 2015 at 7:54 PM, Hyunsik Choi >> wrote: >> >> > >> >> >> This project is looking for mentors. Anyone can help? We are also >> >> >> looking forward to any feedback. >> >> >> >> >> >> Also, I attached the proposal here. I forgot it. >> >> >> >> >> >> ---------------- >> >> >> >> >> >> =3D S2Graph Proposal =3D >> >> >> >> >> >> =3D=3D Abstract =3D=3D >> >> >> S2Graph is a distributed and scalable OLTP graph database built on >> >> >> HBase to support fast traversal on extremely large graph. >> >> >> >> >> >> Here are additional materials to introduce S2Graph. >> >> >> * HBaseCon 2015 - >> >> http://www.slideshare.net/HBaseCon/use-cases-session-5 >> >> >> * Apache: Big Data 2015 - >> >> >> >> http://schd.ws/hosted_files/apachebigdata2015/06/s2graph_apache_con.pdf >> >> >> >> >> >> =3D=3D Proposal =3D=3D >> >> >> S2Graph is to provide a scalable distributed graph database engine >> >> >> over key/value storage such as HBase. S2Graph provide fully >> >> >> ashynchronous API to manupulate data as property graph model and f= ast >> >> >> breadth first search query on graph. >> >> >> >> >> >> =3D=3D Background =3D=3D >> >> >> S2Graph initially started as an internal project at Kakao.com to >> >> >> efficiently store user relation and user activities as one large >> graph >> >> >> and provide unified query to traverse graph. It was open sourced o= n >> >> >> Github about a 3 months ago in June 2015. >> >> >> >> >> >> Over time S2Graph, together with HBase as storage tier, has begun = to >> >> >> be adapted into various applications, such as messaging, social >> feeds, >> >> >> realtime recommendations at Kakao. >> >> >> >> >> >> Users can benefit from S2Graph`s generalized high level API instea= d >> of >> >> >> low-level key/value API for graph abstraction, just like Phoenix >> >> >> provide SQL layer over HBase. >> >> >> >> >> >> =3D=3D Rationale =3D=3D >> >> >> Graph data(highly interconnected data) is very abundant and import= ant >> >> >> these days. >> >> >> When users have a multitude of relationships, each with complex >> >> >> properties associated with them, graph model is more intuitive and >> >> >> efficient than tabular format(RDBMS). >> >> >> There are many ASF projects that provide SQL layer, but there is n= o >> >> >> ASF projects that provide scalable graph layer on existing hadoop >> echo >> >> >> system. >> >> >> When graph data grows to trillion edge scale, the process of >> >> >> traversing takes a long time and costly. However, with the benefit= of >> >> >> HBase`s scalable architecture, S2Graph can traverse large graph in >> >> >> breadth first search manner efficiently. >> >> >> >> >> >> S2Graph also interoperates with several existing Apache >> >> >> projects(HBase, Spark) to provide way to merge real time events an= d >> >> >> batch processed data using property graph data model. >> >> >> >> >> >> Many developers are running their own domain specific API servers = to >> >> >> serve their data products, but graph model is general and S2Graph = API >> >> >> fully support traverse on graph, so it can be used as scalable >> general >> >> >> purpose API serving layer for various domains. >> >> >> As long as data can be modeled as graph, then users can avoid tedi= ous >> >> >> work for developing customized API servers by using S2Graph. >> >> >> >> >> >> =3D=3D Initial Goals =3D=3D >> >> >> The initial goals will be to move the existing codebase to Apache = and >> >> >> integrate with the Apache development process. Once this is >> >> >> accomplished, we plan for incremental development and releases tha= t >> >> >> follow the Apache guidelines. >> >> >> >> >> >> =3D=3D Current Status =3D=3D >> >> >> >> >> >> =3D=3D=3D Meritocracy =3D=3D=3D >> >> >> S2Graph operated on meritocratic principles from the get go. >> >> >> Currently, all the discussions pertaining to S2Graph development a= re >> >> >> public on Github. The current incubation >> >> >> proposal includes the major code contributors to S2Graph. Several >> >> >> additional people have worked on the S2graph codebase for industry >> use >> >> >> cases and would be interested in becoming committers. We are start= ing >> >> >> with a small committer group and we plan to add additional committ= ers >> >> >> following an open merit-based decision process during the incubati= on >> >> >> phase. >> >> >> >> >> >> =3D=3D=3D Community =3D=3D=3D >> >> >> We have already begun building a community but at this time the >> >> >> community consists only of S2Graph developers =E2=80=93 all Kakao = employees =E2=80=93 >> >> >> and prospective users. >> >> >> S2Graph seeks to develop developer and user communities during >> >> incubation. >> >> >> >> >> >> =3D=3D=3D Core Developers =3D=3D=3D >> >> >> S2Graph is currently being designed and developed by 2 engineers f= rom >> >> >> Kakao. - Doyung Yoon, Deawon Jeong. >> >> >> >> >> >> =3D=3D=3D Alignment =3D=3D=3D >> >> >> Our proposed S2Graph effort aligns closely with Apache HBase. The >> >> >> HBase project perimeter is denoted by a simple byte-array based >> >> >> Create, Read, Update, Delete and Scan APIs with no current plans t= o >> >> >> extend beyond this bounds. >> >> >> >> >> >> S2Graph complements this with a higher level API for property grap= h >> >> model. >> >> >> >> >> >> S2Graph was designed to offer scalable distributed graph database >> skin >> >> >> over HBase from the beginning in order to provide property graph >> model >> >> >> and breadth first search, and continue to focus on providing graph >> >> >> model. >> >> >> >> >> >> =3D=3D Known Risks =3D=3D >> >> >> =3D=3D=3D Orphaned Products =3D=3D=3D >> >> >> The core developers of S2Graph team plan to work full time on this >> >> >> project. There is very little risk of S2Graph getting orphaned sin= ce >> >> >> at least one large company (Kakao) is extensively using it in thei= r >> >> >> production HBase clusters. For example, currently there are 20+ us= e >> >> >> cases with more than 1+Trillion edges and 140 million breadth firs= t >> >> >> search query requests per minute using S2Graph in production. >> >> >> We plan to extend and diversify this community further through >> Apache. >> >> >> >> >> >> =3D=3D=3D Inexperience with Open Source =3D=3D=3D >> >> >> The core developers are all active users and followers of open >> source. >> >> >> They are already committers and contributors to the S2Graph Github >> >> >> project. All have been involved with the source code that has been >> >> >> released under an open source license. Though the core set of >> >> >> Developers do not have Apache Open Source experience, there are pl= ans >> >> >> to onboard individuals with Apache open source experience on to th= e >> >> >> project. >> >> >> >> >> >> =3D=3D=3D Homogenous Developers =3D=3D=3D >> >> >> Most committers in this proposal belong to the same institution >> >> >> (Kakao). The engagement of these committers goes well beyond the >> >> >> necessary development to support research, and all committers work= on >> >> >> S2Graph full time. >> >> >> Several people from other institutions are working on and are >> familiar >> >> >> with the S2Graph codebase. We will work to attract them as future >> >> >> committers during the incubation phase, following a merit-based >> >> >> approach. >> >> >> >> >> >> =3D=3D=3D Reliance on Salaried Developers =3D=3D=3D >> >> >> Kakao invested in S2Graph as the distributed graph database soluti= on >> >> >> on top of HBase and some of its key engineers are working full tim= e >> on >> >> >> the project. >> >> >> We look forward to other Apache developers and researchers to >> >> >> contribute to the project. >> >> >> Also key to addressing the risk associated with relying on Salarie= d >> >> >> developers from a single entity is to increase the diversity of th= e >> >> >> contributors and actively lobby for Domain experts in the graph >> >> >> database space to contribute. Apache S2Graph intends to do this. >> >> >> >> >> >> =3D=3D=3D Relationships with Other Apache Products =3D=3D=3D >> >> >> S2Graph has a strong relationship and dependency with Apache Hadoo= p >> >> >> HBase and Spark. >> >> >> Being part of Apache=E2=80=99s Incubation community, could help wi= th a closer >> >> >> collaboration among these two projects and as well as others. >> >> >> >> >> >> In terms of graph processing frameworks, S2Graph and Apache Giraph >> >> >> look similar. However, their goals are apparently different to eac= h >> >> >> other. Giraph aims at analytical batch processing on immutable gra= ph >> >> >> data sets. In contrast, S2Graph is designed for OLTP-like workload= s >> on >> >> >> graph data sets, and S2Graph provides INSERT/UPDATE operations too= . >> >> >> >> >> >> >> >> >> =3D=3D=3D An Excessive Fascination with the Apache Brand =3D=3D=3D >> >> >> S2Graph is proposing to enter incubation at Apache in order to hel= p >> >> >> efforts to diversify the committer-base, not so much to capitalize= on >> >> >> the Apache brand. The S2Graph project is in production use already >> >> >> inside Kakao, but is not expected to be an Kakao product for exter= nal >> >> >> customers. As such, the S2Graph project is not seeking to use the >> >> >> Apache brand as a marketing tool. >> >> >> >> >> >> =3D=3D Documentation =3D=3D >> >> >> Information about S2Graph can be found at >> >> >> https://github.com/kakao/s2graph. The following links provide more >> >> >> information about S2Graph in open source: >> >> >> * S2Graph web site: >> >> https://steamshon.gitbooks.io/s2graph-book/content/ >> >> >> * Codebase at Github: https://github.com/kakao/s2graph >> >> >> * Issue Tracking: https://github.com/kakao/s2graph/issues >> >> >> * User community: https://groups.google.com/forum/#!forum/s2graph >> >> >> >> >> >> =3D=3D Initial Source =3D=3D >> >> >> >> >> >> The S2Graph codebase is currently hosted on Github: >> >> >> https://github.com/kakao/s2graph >> >> >> >> >> >> =3D=3D=3D Source and Intellectual Property Submission Plan =3D=3D= =3D >> >> >> >> >> >> Currently, the S2Graph codebase is distributed under the Apache 2.= 0 >> >> >> License. >> >> >> >> >> >> =3D=3D External Dependencies =3D=3D >> >> >> >> >> >> Beyond relying on Apache HBase, Phoenix has the following external >> >> >> dependencies: >> >> >> * Asynchbase (BSD license: http://www.antlr3.org/license.html) >> >> >> * Mysql (BSD license: >> >> >> https://github.com/julianhyde/sqlline/blob/master/LICENSE) >> >> >> * Play Framework (Apache 2.0 license: >> >> >> https://github.com/playframework/playframework) >> >> >> * Scala (https://github.com/scala/scala) >> >> >> * Spark >> >> >> * Kafka >> >> >> >> >> >> =3D=3D Required Resources =3D=3D >> >> >> >> >> >> =3D=3D=3D Mailing list =3D=3D=3D >> >> >> >> >> >> We will migrate our mailing lists to the following: >> >> >> * users@s2graph.incubator.apache.org >> >> >> * dev@s2graph.incubator.apache.org >> >> >> * private@s2graph.incubator.apache.org >> >> >> * commits@s2graph.incubator.apache.org >> >> >> >> >> >> =3D=3D=3D Source control =3D=3D=3D >> >> >> >> >> >> The S2Graph team would like to use Git for source control, due to = our >> >> >> current use of Git. We request a writeable Git repo for S2Graph, a= nd >> >> >> mirroring to be set up to Github through INFRA. >> >> >> >> >> >> =3D=3D=3D Issue Tracking =3D=3D=3D >> >> >> >> >> >> S2Graph currently uses the github issue tracking system associated >> >> >> with its github repo: https://github.com/kakao/s2graph/issues. We >> will >> >> >> migrate to the Apache JIRA: >> >> >> http://issues.apache.org/jira/browse/S2Graph >> >> >> >> >> >> =3D=3D=3D Other Resources =3D=3D=3D >> >> >> >> >> >> Jenkins/Hudson for builds and test running. >> >> >> Wiki for documentation purposes >> >> >> Blog to improve project dissemination >> >> >> >> >> >> =3D=3D Initial Committers =3D=3D >> >> >> >> >> >> * Doyung Yoon >> >> >> * Daewon Jeong >> >> >> * Jaesang Kim >> >> >> * Hwansung Yu >> >> >> * Min-Seok Kim >> >> >> * Chul Kang >> >> >> >> >> >> =3D=3D Affiliations =3D=3D >> >> >> >> >> >> The initial committers are from one organizations: Kakao. >> >> >> * Doyung Yoon, Kakao >> >> >> * Daewon Jeong, Kakao >> >> >> * Jaesang Kim, Kakao >> >> >> * Hwansung Yu, Kakao >> >> >> * Min-Seok Kim, Kakao >> >> >> * Chul Kang, Kakao >> >> >> >> >> >> =3D=3D Sponsors =3D=3D >> >> >> >> >> >> =3D=3D=3D Champion =3D=3D=3D >> >> >> Hyunsik Choi >> >> >> >> >> >> =3D=3D=3D Nominated Mentors =3D=3D=3D >> >> >> >> >> >> =3D=3D=3D Sponsoring Entity =3D=3D=3D >> >> >> >> >> >> * The Apache Incubator >> >> >> >> >> >> On Fri, Nov 6, 2015 at 4:05 PM, Hyunsik Choi >> >> wrote: >> >> >> > Hi Seetharam, >> >> >> > >> >> >> > Thank you for a good question. That seem to be a frequent questi= on >> to >> >> >> > this project. >> >> >> > >> >> >> > Here is the answer to your question. >> >> >> > >> >> >> >> >> >> https://steamshon.gitbooks.io/s2graph-book/content/what_is_different_to_= titan.html >> >> >> > >> >> >> > I hope that this link is helpful to your understanding. >> >> >> > >> >> >> > Best regards, >> >> >> > Hyunsik >> >> >> > >> >> >> > >> >> >> > >> >> >> > On Fri, Nov 6, 2015 at 3:07 PM, Seetharam Venkatesh >> >> >> > wrote: >> >> >> >> Hi Hyunsik, >> >> >> >> >> >> >> >> The proposal looks interesting and want to know how is this >> different >> >> >> from >> >> >> >> existing solutions in the same space such as Titan, etc. >> >> >> >> >> >> >> >> Thanks! >> >> >> >> Venkatesh >> >> >> >> >> >> >> >> >> >> >> >> On Fri, Nov 6, 2015 at 1:36 PM Hyunsik Choi >> >> wrote: >> >> >> >> >> >> >> >>> Hi folks, >> >> >> >>> >> >> >> >>> We would like to start a discussion on S2Graph as an incubatio= n >> >> >> project. >> >> >> >>> >> >> >> >>> S2Graph is a distributed and scalable OLTP graph database buil= t >> on >> >> >> >>> HBase. It provides interactive queries for >> vertex/edge/sub-graphs on >> >> >> >>> extremely large graph data sets as well as insertion and updat= e >> >> >> >>> operations. >> >> >> >>> >> >> >> >>> S2Graph was already introduced in Apache BigData and HBaseCon >> this >> >> >> year. >> >> >> >>> >> >> >> >>> The proposal is available at : >> >> >> >>> https://wiki.apache.org/incubator/S2GraphProposal >> >> >> >>> >> >> >> >>> We are looking forward to any feedback. In addition, we are >> looking >> >> >> >>> for volunteers as mentors. >> >> >> >>> >> >> >> >>> Best regards, >> >> >> >>> Hyunsik >> >> >> >>> >> >> >> >>> >> >> --------------------------------------------------------------------- >> >> >> >>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.o= rg >> >> >> >>> For additional commands, e-mail: >> general-help@incubator.apache.org >> >> >> >>> >> >> >> >>> >> >> >> >> >> >> ------------------------------------------------------------------= --- >> >> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >> >> >> For additional commands, e-mail: general-help@incubator.apache.org >> >> >> >> >> >> >> >> > >> >> > >> >> > -- >> >> > Sergio Fern=C3=A1ndez >> >> > Partner Technology Manager >> >> > Redlink GmbH >> >> > m: +43 6602747925 >> >> > e: sergio.fernandez@redlink.co >> >> > w: http://redlink.co >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >> >> For additional commands, e-mail: general-help@incubator.apache.org >> >> >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >> For additional commands, e-mail: general-help@incubator.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org For additional commands, e-mail: general-help@incubator.apache.org