From dev-return-53966-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Tue Sep 18 18:09:27 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4A159180677 for ; Tue, 18 Sep 2018 18:09:26 +0200 (CEST) Received: (qmail 65230 invoked by uid 500); 18 Sep 2018 16:09:25 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 65218 invoked by uid 99); 18 Sep 2018 16:09:24 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Sep 2018 16:09:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 4FCF91842E5 for ; Tue, 18 Sep 2018 16:09:24 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.869 X-Spam-Level: * X-Spam-Status: No, score=1.869 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Nvo2OPiwaJiS for ; Tue, 18 Sep 2018 16:09:21 +0000 (UTC) Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 3DF585F418 for ; Tue, 18 Sep 2018 16:09:21 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id d19-v6so1277369pgv.1 for ; Tue, 18 Sep 2018 09:09:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=US9AIxtP7jWwbrOWIcU0iQu+mu5X+hFQ0Ap0q9uM2KY=; b=UM7hnsx7BcG12K6ZxV4GRdw6nxnVSh7XU/5AlTGF48msnlUuBdr7YgpHr9e62ypcni IPS2mGxcoyItaGR0UmBOdcVR0ArEfQ/ALg983Z6JHvX5c3HD1RVhdqp5z6JL1iTj2AQ/ fYD3/IWu2jZ+xIVmcEEW3wlYsHJd98jyVstmYek/Ed9FG0gzIt6e6PfN1XVimRcR4mKg vUlnVxLsFsGYM8DGNNuPCDM3OV9I1T5T1UPILsMgQ/mYGmo09ltXibmC73hG9n5lu5a/ DlFD5SMO9Ra/pl5HGr4K2UY0puhjewEsj4pnxpM66mVmBTjlPs8kZXQaMlb5afqX7IVP LHpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=US9AIxtP7jWwbrOWIcU0iQu+mu5X+hFQ0Ap0q9uM2KY=; b=eUNXPLnIAOpMLwNRitLdpITVTnVUmpQ2pnFKSrd0uDP88pjsu/RToYHwhXYVYfEN5h H3N5cJpbvs0wHPAHhgUKMgL0KWoPtt52WExkWW24THKwcs9ugh+NYArPc681KtYQ9r7Z Dhs8H8wQu9NhRl7+E0o3HDxXiG0/I2bGjwUuLNXfyiKgT+89lWtyxEdSImwMb5x+hb5K SAe3q+p4xxghIyfq6NEnrGLJlyA/QtX0KX7dyqf5+MBG5+8WMDSJ3Jd3gkmd5aG46uMr Pxsw6WdWvqItdXeBpPNaw0AFejWdQRSaJTHW/HMso/hu3maGZfbrRiJ3oueJXeL+9sDH JwFw== X-Gm-Message-State: APzg51DBAvE/yUuBz+P/QbRxiCPsS3yBM+Vmw1WnpTaEMYkhKdcCdA/5 4076so8/EDGu/o8kIkOwkhagLDfn/e2+Pt4ADbUObXvgeTY= X-Google-Smtp-Source: ANB0VdYn2vfQbKccs8UUFAfbMyuoZjc0EAMjf9uI6inT9LCme/5RgBy/KWPeyCAKZYOMs+HqLZ5OnqLovS9w+OnQj5Q= X-Received: by 2002:a63:25c4:: with SMTP id l187-v6mr28577227pgl.29.1537286959853; Tue, 18 Sep 2018 09:09:19 -0700 (PDT) MIME-Version: 1.0 References: <55D8D1CF-54E4-4D44-A5AF-DD233940314C@gmail.com> In-Reply-To: From: Jaanai Zhang Date: Wed, 19 Sep 2018 00:08:53 +0800 Message-ID: Subject: Re: [DISCUSS] Suggestions for Phoenix from HBaseCon Asia notes To: dev@phoenix.apache.org Content-Type: multipart/alternative; boundary="00000000000028f4e60576278346" --00000000000028f4e60576278346 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > > I don't understand what performance issues you think exist based solely o= n > the above. Those numbers appear to be precisely in line with my > expectations. Can you please describe what issues you think exist? > 1. Performance of the thick client has almost 1~4 time higher than the thin client, the performance of the thin client will be decreased when the number of concurrencies is increased. For some applications of the web server, this is not enough. 2. An HA thin client. 3. SQL audit function A lot of developer like using the thin client, which has a lower maintenance cost on the client.Sorry, that's all that comes to me. :) Please be more specific. Asking for "more documentation" doesn't help us > actually turn this around into more documentation. What are the specific > pain points you have experienced? What topics do you want to know more > about? Be as specific as possible. > About documents: 1. I think that we cloud add documents about migrate tools and migrate cases since many users migrate from RDBMS(MYSQL/PG/SQL SERVER) to Phoenix for some applications of non-transaction. 2. How to design PK or indexes. About pain points: The stability is a big problem. Most of the people use Phoenix as a common RDBMS, they are informal to execute a query, even if they don't know why server crash when a scan full table was executed, so define use boundary of Phoenix is important that rejects some query and reports it user's client. Are you referring to the hbase-spark (and thus, Spark SQL) integration? Or > something that some company is building? > Some companies are building with SPARK SQL to access Phoenix to support OLAP and OLTP requirements. it will produce heavily load for HBase cluster when Spark reads Phoenix tables, my co-workers want to directly read HFiles of Phoenix tables for some offline business, but that depends on more flexible Phoenix API. Uh, I also got some feedback that some features import for users, For example, "alter table modify column" can avoid reloaded data again which is expensive operate for the massive data table. I had upload patches to JIRA( PHOENIX-4815 ), but nobody responds to me :(. Now, i devote to develop chaos test and PQS stability(it was developed on the branch of my company, these patches will contribute to the community after stable running ), if you have any suggests, please tell to me what you thinking. I would appreciate your reply. ---------------------------------------- Jaanai Zhang Best regards! Josh Elser =E4=BA=8E2018=E5=B9=B49=E6=9C=8818=E6=97= =A5=E5=91=A8=E4=BA=8C =E4=B8=8A=E5=8D=8812:03=E5=86=99=E9=81=93=EF=BC=9A > > > On Mon, Sep 17, 2018 at 9:36 AM zhang yun wrote: > >> Sorry for replying late. I attended HBaesCon Asia as a speaker and got >> some some notes. I think Phoenix=E2=80=99 pains as following: >> >> 1. Thick client isn=E2=80=99t as more popular as thin client. For some w= e >> applications: 1. Users need to spend a lot of time to solve the >> dependencies, 2. Users worry about the stability which some calculation >> operates are processed within thick client . 3. Some people hope use mul= ti >> program language client, such as Go, .Net and Python etc=E2=80=A6 Other= benefits: >> 1 Easy to add SQL audit function. 2. Recognise invalid SQL and report t= o >> user... As you said this definitely a big issue which is worth paying >> more attention. However thick client exists some problems, it is recen= tly >> test data about performance: >> >> > I don't understand what performance issues you think exist based solely o= n > the above. Those numbers appear to be precisely in line with my > expectations. Can you please describe what issues you think exist? > > >> 2. Actually, Phoenix has a high barrier for beginner than common RDBMS, >> users need to learn HBase before using Phoenix, Most of people don=E2=80= =99t know >> how to reasonable to use, so we need more detail documents make Phoeni= x >> use easier. >> > > Please be more specific. Asking for "more documentation" doesn't help us > actually turn this around into more documentation. What are the specific > pain points you have experienced? What topics do you want to know more > about? Be as specific as possible. > > >> 3. HBase 3.0 has a plan about native SQL, does Phoenix has a plan? Even >> if many peoples don=E2=80=99t know HBase has a SQL layer which is called= Phoenix, >> so can we put the link on HBase website? >> > > Uh, I have no idea what you're referring to here about "native SQL". I am > not aware of any such effort that does this solely inside of HBase, nor > does it seem inline with HBase's "do one thing well" mantra. > > Are you referring to the hbase-spark (and thus, Spark SQL) integration? O= r > something that some company is building? > > How about submitting patch to HBase to modify > https://hbase.apache.org/poweredbyhbase.html ? :) > > >> >> On 2018/08/27 18:03:30, Josh Elser wrote: >> > (bcc: dev@hbase, in case folks there have been waiting for me to send >> > >> > this email to dev@phoenix)> >> > >> > Hi,> >> > >> > In case you missed it, there was an HBaseCon event held in Asia > >> > recently. Stack took some great notes and shared them with the HBase > >> > community. A few of them touched on Phoenix, directly or in a related = > >> > manner. I think they are good "criticisms" that are beneficial for us >> to > >> > hear.> >> > >> > 1. The phoenix-$version-client.jar size is prohibitively large> >> > >> > In this day and age, I'm surprised that this is a big issue for people= . >> > >> > I know have a lot of cruft, most of which coming from hadoop. We have = > >> > gotten better here over recent releases, but I would guess that there >> is > >> > more we can do.> >> > >> > 2. Can Phoenix be the de-facto schema for SQL on HBase?> >> > >> > We've long asserted "if you have to ask how Phoenix serializes data, >> you > >> > shouldn't be do it" (a nod that you have to write lots of code). What >> if > >> > we turn that on its head? Could we extract our PDataType serialization= , >> > >> > composite row-key, column encoding, etc into a minimal API that folks = > >> > with their own itches can use?> >> > >> > With the growing integrations into Phoenix, we could embrace them by > >> > providing an API to make what they're doing easier. In the same vein, >> we > >> > cement ourselves as a cornerstone of doing it "correctly".> >> > >> > 3. Better recommendations to users to not attempt certain queries.> >> > >> > We definitively know that there are certain types of queries that > >> > Phoenix cannot support well (compared to optimal Phoenix use-cases). > >> > Users very commonly fall into such pitfalls on their own and this >> leaves > >> > a bad taste in their mouth (thinking that the product "stinks").> >> > >> > Can we do a better job of telling the user when and why it happened? > >> > What would such a user-interaction model look like? Can we supplement = > >> > the "why" with instructions of what to do differently (even if in the = > >> > abstract)?> >> > >> > 4. Phoenix-Calcite> >> > >> > This was mentioned as a "nice to have". From what I understand, there = > >> > was nothing explicitly from with the implementation or approach, just = > >> > that it was a massive undertaking to continue with little immediate > >> > gain. Would this be a boon for us to try to continue in some form? Are >> > >> > there steps we can take that would help push us along the right path?> >> > >> > Anyways, I'd love to hear everyone's thoughts. While the concerns were >> > >> > raised at HBaseCon Asia, the suggestions that accompany them here are = > >> > largely mine ;). Feel free to break them out into their own threads if >> > >> > you think that would be better (or say that you disagree with me -- > >> > that's cool too)!> >> > >> > - Josh> >> > >> > --00000000000028f4e60576278346--