From dev-return-53814-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Thu Aug 30 19:20:43 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 04F50180656 for ; Thu, 30 Aug 2018 19:20:42 +0200 (CEST) Received: (qmail 10593 invoked by uid 500); 30 Aug 2018 17:20:42 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 10581 invoked by uid 99); 30 Aug 2018 17:20:41 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Aug 2018 17:20:41 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 10B2F1A07FF for ; Thu, 30 Aug 2018 17:20:41 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.888 X-Spam-Level: * X-Spam-Status: No, score=1.888 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=salesforce.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id oJnikcYwIDzd for ; Thu, 30 Aug 2018 17:20:39 +0000 (UTC) Received: from mail-lj1-f182.google.com (mail-lj1-f182.google.com [209.85.208.182]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 1A87B5F36D for ; Thu, 30 Aug 2018 17:20:39 +0000 (UTC) Received: by mail-lj1-f182.google.com with SMTP id v26-v6so7905691ljj.3 for ; Thu, 30 Aug 2018 10:20:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salesforce.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=8WjuPy+ny0C7L/2YCk4+aZCW+rvC5+i7tTAzDs/dgGw=; b=C1ji0Ok4N3BuVhn1kRRkvx8ZVkwwGA5bwdevG4qhSeZu8G5Prn+Aa2PSbzy32mfte7 cdvQdSuIl7aupEO3FbHiL5siVxw71lqNQjhTj3sTQt3lvZVOeZQPukukTBxf2bDEF0dR 6jMN5Dmmv5xvIbxCwVJXXREMQGrJO61LIdzPs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=8WjuPy+ny0C7L/2YCk4+aZCW+rvC5+i7tTAzDs/dgGw=; b=AarC0PXSjnUw1rqIhGLO3c6y5ite0B5B8Gfojti8hk7Hn+3BuQioIs29lK9p5LR69F Xy2Umln8gxCOWBqkSynNmK+7vOP7AhuD6hSqzKekARsrAxsXOxauWLyLqHkQtvwhOcS9 z3hjebImUAzFhFKdRYTwHjQvaeWrfJ6YEbEOgfBKBEIkHX+vGbFcUt3wIPynL82Wi2sI iDmc1ZZoKmnMHkd169ZWQZQLkFTAz2MEOkcXRtfyHAQtQRV7lxD4JsunVvH683oWdS2u aYG9ispgFmQnFvuIjJAh3OZaQ5he89MmQltXWOtyxQxUEoo0585H7UtLx+yXeD/hGLoA J1Ww== X-Gm-Message-State: APzg51CsdEQaM/OtAJ8a+SZtbMA90DrxlS0p4NrpJvDyY8/OST48sbha 3h6SvtpM89iqhBv3EQp4XNW2M/5F6gQRiSq1p/JkoSruXq4= X-Google-Smtp-Source: ANB0VdYdI6lxW85D1Ch0j7JsukxQoMBLn2nCbPQldEopE1eOrMKuRFKQxq6WhJJ2kIxnnvOcYCsxD4F3KdU4Znb0k+4= X-Received: by 2002:a2e:6c17:: with SMTP id h23-v6mr7690806ljc.81.1535649638123; Thu, 30 Aug 2018 10:20:38 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:888a:0:0:0:0:0 with HTTP; Thu, 30 Aug 2018 10:20:37 -0700 (PDT) In-Reply-To: References: From: "Thomas D'Silva" Date: Thu, 30 Aug 2018 10:20:37 -0700 Message-ID: Subject: Re: [DISCUSS] EXPLAIN'ing what we do well (was Re: [DISCUSS] Suggestions for Phoenix from HBaseCon Asia notes) To: dev@phoenix.apache.org Content-Type: multipart/alternative; boundary="0000000000002e64590574aa4bf4" --0000000000002e64590574aa4bf4 Content-Type: text/plain; charset="UTF-8" I created PHOENIX-4881 to create a guardrail config property based on the bytes scanned. We already have PHOENIX-1481 to improve the explain plan documentation. On Tue, Aug 28, 2018 at 1:40 PM, James Taylor wrote: > Thomas' idea is a good one. From the EXPLAIN plan ResultSet, you can > directly get an estimate of the number of bytes that will be scanned. Take > a look at this [1] documentation. We need to implement PHOENIX-4735 too (so > that things are setup well out-of-the-box). We could have a kind of > guardrail config property that would define the max allowed bytes allowed > to be read and fail a query that goes over this limit. That would cover 80% > of the issues IMHO. Other guardrail config properties could cover other > corner cases. > > [1] http://phoenix.apache.org/explainplan.html > > On Mon, Aug 27, 2018 at 3:01 PM Josh Elser wrote: > > > On 8/27/18 5:03 PM, Thomas D'Silva wrote: > > >> 3. Better recommendations to users to not attempt certain queries. > > >> > > >> We definitively know that there are certain types of queries that > > Phoenix > > >> cannot support well (compared to optimal Phoenix use-cases). Users > very > > >> commonly fall into such pitfalls on their own and this leaves a bad > > taste > > >> in their mouth (thinking that the product "stinks"). > > >> > > >> Can we do a better job of telling the user when and why it happened? > > What > > >> would such a user-interaction model look like? Can we supplement the > > "why" > > >> with instructions of what to do differently (even if in the abstract)? > > >> > > > Providing relevant feedback before/after a query is run in general is > > very > > > hard to do. If stats are enabled we have an estimate of how many > > rows/bytes > > > will be scanned. > > > We could have an optional feature that prevent users from running > queries > > > if the rows/bytes scanned are above a certain threshold. We should also > > > enhance our explain > > > plan documentationhttp://phoenix.apache.org/explainplan.html with > > example > > > of queries so users know what kinds of queries Phoenix handles well. > > > > Breaking this out.. > > > > Totally agree -- this is by no means "easy". I struggle very often > > trying to express just _why_ a query that someone is running in Phoenix > > doesn't run as well as they think it should. > > > > Centralizing on the EXPLAIN plan is good. Making sure it's > > consumable/thorough is probably the lowest hanging fruit. If we can give > > concrete examples to the kinds of explain plans a user might see, I > > think that might get use from users/admins. > > > > Throwing a random idea out there: with stats and the query plan, can we > > give a thumbs-up/thumbs-down? If we can, is that useful? > > > --0000000000002e64590574aa4bf4--