From dev-return-53973-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Wed Sep 19 13:49:17 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id AC217180621 for ; Wed, 19 Sep 2018 13:49:16 +0200 (CEST) Received: (qmail 71062 invoked by uid 500); 19 Sep 2018 11:49:15 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 71049 invoked by uid 99); 19 Sep 2018 11:49:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Sep 2018 11:49:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id AB98BC208E for ; Wed, 19 Sep 2018 11:49:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.889 X-Spam-Level: * X-Spam-Status: No, score=1.889 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id v_-C_NSaftzn for ; Wed, 19 Sep 2018 11:48:27 +0000 (UTC) Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 8D6275FB8D for ; Wed, 19 Sep 2018 11:48:27 +0000 (UTC) Received: by mail-pg1-f169.google.com with SMTP id t84-v6so2640366pgb.5 for ; Wed, 19 Sep 2018 04:48:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=ox36cEhhatO5J4g3ynLe9oX5aYbs2bLdgEQ1Cud+9y0=; b=RNAnmhPhGFzU/FPTM9oQo0PusiioA/hG05Af5EMbpuct0+5O7pcGxaRVJ6dOv6xKL1 3THgYI7fciJzhl0kbts2sXt8k8BRRUIsJe4Pqli5WyH6T3Viza+zUKsZjAfbtMBb7tDu U0IKDEb+44lS30J5XFv82+8nZDO57iV3GSS4bw1zmIxBmhRfCw2YNyah8FtddQ+MCPyh pKfNMAeIXeZfgnKvXLW260+h5QaNcpooEGkg8iTvIKxztUwhlYI+Y+M8keT50C/d7Z0/ A16jYHMDfXtXatHCoR4sR/N55d3+iYFmfELWL5wnNYNjOvKpawxXnepw546WDqCBe7lf l6lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ox36cEhhatO5J4g3ynLe9oX5aYbs2bLdgEQ1Cud+9y0=; b=F04AXTn0C2feZ+e0Nl/tCHs3zeEq1RVujKPGoGwGO8krJs/WjfYsrghoNkhJk+gGE6 r/63uHNPQEKH2+z/JUTl/stQpWN90s5osUH62xA4Zy35wwksWbyhFH4KQFBW1CsCaSdS 7Q91UasBB7Yx37SxR309vgTDdYQOje2MimpQSJkOr/rNB40H25NjQX6Qg6DLQugbqxy3 NnJzcxFq8/WTthkWH2uwWaZgKGFzRLK83FBb4L3WqwlMzPjjKwJMebRATXLD3eNs+ejo sWaSn5culTvHkmlfbPJRFEtCPcvT140S9sRHSkXppvkrqPbdDfly4PRURx5AhxxppqUQ fe7Q== X-Gm-Message-State: APzg51AqrD95jEySL/qeQMNJyAerPGil2CDFWc/roBVzoaXBO/LseMIp gpcXOhAr8ONeLbQMlAjZ9gydlm1rFJSQwGfy7SsGbMUXP0g= X-Google-Smtp-Source: ANB0VdZ0qpTrlM/4+WnWnyxjA3TbVfnzPwhUd4czgpqd+jkn4IVNyTL4p81PvqPxOtEXkXPOUtw5/cKyB82wDIJZneE= X-Received: by 2002:a65:5089:: with SMTP id r9-v6mr2042352pgp.216.1537357699359; Wed, 19 Sep 2018 04:48:19 -0700 (PDT) MIME-Version: 1.0 References: <55D8D1CF-54E4-4D44-A5AF-DD233940314C@gmail.com> In-Reply-To: From: Jaanai Zhang Date: Wed, 19 Sep 2018 19:47:53 +0800 Message-ID: Subject: Re: [DISCUSS] Suggestions for Phoenix from HBaseCon Asia notes To: dev@phoenix.apache.org Content-Type: multipart/alternative; boundary="0000000000009029c6057637fbc0" --0000000000009029c6057637fbc0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > > How about submitting patch to HBase to modify > https://hbase.apache.org/poweredbyhbase.html ? :) It will be hard to find by folks if we add Phoneix's link to this page. May the home page of HBase is a good place... but which needs the approval of HBase community. ---------------------------------------- Jaanai Zhang Best regards! Jaanai Zhang =E4=BA=8E2018=E5=B9=B49=E6=9C=8819=E6= =97=A5=E5=91=A8=E4=B8=89 =E4=B8=8A=E5=8D=8812:08=E5=86=99=E9=81=93=EF=BC=9A > I don't understand what performance issues you think exist based solely o= n >> the above. Those numbers appear to be precisely in line with my >> expectations. Can you please describe what issues you think exist? >> > > 1. Performance of the thick client has almost 1~4 time higher than the > thin client, the performance of the thin client will be decreased when th= e > number of concurrencies is increased. For some applications of the web > server, this is not enough. > 2. An HA thin client. > 3. SQL audit function > > A lot of developer like using the thin client, which has a lower > maintenance cost on the client.Sorry, that's all that comes to me. :) > > Please be more specific. Asking for "more documentation" doesn't help us >> actually turn this around into more documentation. What are the specific >> pain points you have experienced? What topics do you want to know more >> about? Be as specific as possible. >> > > About documents: > 1. I think that we cloud add documents about migrate tools and migrate > cases since many users migrate from RDBMS(MYSQL/PG/SQL SERVER) to Phoenix > for some applications of non-transaction. > 2. How to design PK or indexes. > > About pain points: > The stability is a big problem. Most of the people use Phoenix as a commo= n > RDBMS, they are informal to execute a query, even if they don't know why > server crash when a scan full table was executed, so define use boundary = of > Phoenix is important that rejects some query and reports it user's client= . > > Are you referring to the hbase-spark (and thus, Spark SQL) integration? O= r >> something that some company is building? >> > > Some companies are building with SPARK SQL to access Phoenix to support > OLAP and OLTP requirements. it will produce heavily load for HBase cluste= r > when Spark reads Phoenix tables, my co-workers want to directly read > HFiles of Phoenix tables for some offline business, but that depends on > more flexible Phoenix API. > > Uh, I also got some feedback that some features import for users, For > example, "alter table modify column" can avoid reloaded data again which = is > expensive operate for the massive data table. I had upload patches to JIR= A( > PHOENIX-4815 ), but > nobody responds to me :(. > > Now, i devote to develop chaos test and PQS stability(it was developed o= n > the branch of my company, these patches will contribute to the community > after stable running ), if you have any suggests, please tell to me what > you thinking. I would appreciate your reply. > > > ---------------------------------------- > Jaanai Zhang > Best regards! > > > > Josh Elser =E4=BA=8E2018=E5=B9=B49=E6=9C=8818=E6= =97=A5=E5=91=A8=E4=BA=8C =E4=B8=8A=E5=8D=8812:03=E5=86=99=E9=81=93=EF=BC=9A > >> >> >> On Mon, Sep 17, 2018 at 9:36 AM zhang yun wrote= : >> >>> Sorry for replying late. I attended HBaesCon Asia as a speaker and got >>> some some notes. I think Phoenix=E2=80=99 pains as following: >>> >>> 1. Thick client isn=E2=80=99t as more popular as thin client. For some = we >>> applications: 1. Users need to spend a lot of time to solve the >>> dependencies, 2. Users worry about the stability which some calculation >>> operates are processed within thick client . 3. Some people hope use mu= lti >>> program language client, such as Go, .Net and Python etc=E2=80=A6 Othe= r benefits: >>> 1 Easy to add SQL audit function. 2. Recognise invalid SQL and report = to >>> user... As you said this definitely a big issue which is worth payin= g >>> more attention. However thick client exists some problems, it is rece= ntly >>> test data about performance: >>> >>> >> I don't understand what performance issues you think exist based solely >> on the above. Those numbers appear to be precisely in line with my >> expectations. Can you please describe what issues you think exist? >> >> >>> 2. Actually, Phoenix has a high barrier for beginner than common RDBMS, >>> users need to learn HBase before using Phoenix, Most of people don=E2= =80=99t know >>> how to reasonable to use, so we need more detail documents make Phoen= ix >>> use easier. >>> >> >> Please be more specific. Asking for "more documentation" doesn't help us >> actually turn this around into more documentation. What are the specific >> pain points you have experienced? What topics do you want to know more >> about? Be as specific as possible. >> >> >>> 3. HBase 3.0 has a plan about native SQL, does Phoenix has a plan? Even >>> if many peoples don=E2=80=99t know HBase has a SQL layer which is calle= d Phoenix, >>> so can we put the link on HBase website? >>> >> >> Uh, I have no idea what you're referring to here about "native SQL". I a= m >> not aware of any such effort that does this solely inside of HBase, nor >> does it seem inline with HBase's "do one thing well" mantra. >> >> Are you referring to the hbase-spark (and thus, Spark SQL) integration? >> Or something that some company is building? >> >> How about submitting patch to HBase to modify >> https://hbase.apache.org/poweredbyhbase.html ? :) >> >> >>> >>> On 2018/08/27 18:03:30, Josh Elser wrote: >>> > (bcc: dev@hbase, in case folks there have been waiting for me to send >>> > >>> > this email to dev@phoenix)> >>> > >>> > Hi,> >>> > >>> > In case you missed it, there was an HBaseCon event held in Asia > >>> > recently. Stack took some great notes and shared them with the HBase = > >>> > community. A few of them touched on Phoenix, directly or in a related >>> > >>> > manner. I think they are good "criticisms" that are beneficial for us >>> to > >>> > hear.> >>> > >>> > 1. The phoenix-$version-client.jar size is prohibitively large> >>> > >>> > In this day and age, I'm surprised that this is a big issue for >>> people. > >>> > I know have a lot of cruft, most of which coming from hadoop. We have >>> > >>> > gotten better here over recent releases, but I would guess that there >>> is > >>> > more we can do.> >>> > >>> > 2. Can Phoenix be the de-facto schema for SQL on HBase?> >>> > >>> > We've long asserted "if you have to ask how Phoenix serializes data, >>> you > >>> > shouldn't be do it" (a nod that you have to write lots of code). What >>> if > >>> > we turn that on its head? Could we extract our PDataType >>> serialization, > >>> > composite row-key, column encoding, etc into a minimal API that folks >>> > >>> > with their own itches can use?> >>> > >>> > With the growing integrations into Phoenix, we could embrace them by = > >>> > providing an API to make what they're doing easier. In the same vein, >>> we > >>> > cement ourselves as a cornerstone of doing it "correctly".> >>> > >>> > 3. Better recommendations to users to not attempt certain queries.> >>> > >>> > We definitively know that there are certain types of queries that > >>> > Phoenix cannot support well (compared to optimal Phoenix use-cases). = > >>> > Users very commonly fall into such pitfalls on their own and this >>> leaves > >>> > a bad taste in their mouth (thinking that the product "stinks").> >>> > >>> > Can we do a better job of telling the user when and why it happened? = > >>> > What would such a user-interaction model look like? Can we supplement >>> > >>> > the "why" with instructions of what to do differently (even if in the >>> > >>> > abstract)?> >>> > >>> > 4. Phoenix-Calcite> >>> > >>> > This was mentioned as a "nice to have". From what I understand, there >>> > >>> > was nothing explicitly from with the implementation or approach, just >>> > >>> > that it was a massive undertaking to continue with little immediate > >>> > gain. Would this be a boon for us to try to continue in some form? Ar= e >>> > >>> > there steps we can take that would help push us along the right path?= > >>> > >>> > Anyways, I'd love to hear everyone's thoughts. While the concerns wer= e >>> > >>> > raised at HBaseCon Asia, the suggestions that accompany them here are >>> > >>> > largely mine ;). Feel free to break them out into their own threads i= f >>> > >>> > you think that would be better (or say that you disagree with me -- > >>> > that's cool too)!> >>> > >>> > - Josh> >>> > >>> >> --0000000000009029c6057637fbc0--