From dev-return-53926-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Thu Sep 13 18:33:16 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id C4E98180649 for ; Thu, 13 Sep 2018 18:33:15 +0200 (CEST) Received: (qmail 6790 invoked by uid 500); 13 Sep 2018 16:33:14 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 6778 invoked by uid 99); 13 Sep 2018 16:33:14 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Sep 2018 16:33:14 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id D18211804F8 for ; Thu, 13 Sep 2018 16:33:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.888 X-Spam-Level: * X-Spam-Status: No, score=1.888 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=salesforce.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id NoHUwYlWxbEv for ; Thu, 13 Sep 2018 16:33:12 +0000 (UTC) Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 4A41B5F23A for ; Thu, 13 Sep 2018 16:33:12 +0000 (UTC) Received: by mail-lf1-f43.google.com with SMTP id h64-v6so5340498lfi.10 for ; Thu, 13 Sep 2018 09:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salesforce.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=npO0hj1lJkQD0OLICv8QsB7Z8UI4LzPElrXk0T1FXcc=; b=JeluvH//t+c7IALgSFt78y4KovJcPjTpc2//I7OM6gAbBjEm6kJo60eJZOo8o63Dm4 aRNy6rqlY30XSFF140Tyrnz5IgbDDO9xuLN+LdXzV5KYYLuL5Knac74bPJnkN1p+153L jMjYTzLsW0Jey346rOiG+iJbVGNypN3Mjxjw0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=npO0hj1lJkQD0OLICv8QsB7Z8UI4LzPElrXk0T1FXcc=; b=LqOI8KYuDnJ03+wRg3rmrLcz8yrDrHuPF7vsWMifmL/6432YgPLRuur9ifg+7oSwQN Ou7CQn5aPmTAwPq/Iu8HXayJFoFvsL62KikCnPqo/z2HfhiWMtO7fWHuNsyztmZYiXG/ JJ7WvTLkZRhvdr3bCyi5hyMvH5KcjTHseoXmqsMJNM89+e2hs7KJbUFlly/ErCclz7eB kQlNdU92/6lxsoQDPduHGTVgxPe2Ox1o76Ix7iXvVIizA5FXQ+/9wCwLqN7hd17qvbto 3oeYTwwFb4LpqECuyRHA53Y1p4TfkUPp6nwh+mllPpmb9jtL6xkNjjNNSyPcfG8g/6tB yAXw== X-Gm-Message-State: APzg51AICGL33r7rghwj5OzpBsV2nMZewzIqRvDbuT2RYvonS4xGrLP5 UxmX8dyM0jbEhQgtYIX+UnelTKwCiAzNabvGncXfnw== X-Google-Smtp-Source: ANB0VdYLWs81ZJahuMobZ0TGU+U/iXwpBsPXveJNpBcuY2ma1pqG8K5Kf4fbSQP0AM6pqqQhdlBGznC+m5z2UlfmhPg= X-Received: by 2002:a19:5154:: with SMTP id f81-v6mr5679920lfb.55.1536856385345; Thu, 13 Sep 2018 09:33:05 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:65c6:0:0:0:0:0 with HTTP; Thu, 13 Sep 2018 09:33:04 -0700 (PDT) In-Reply-To: References: From: "Thomas D'Silva" Date: Thu, 13 Sep 2018 09:33:04 -0700 Message-ID: Subject: Re: Salting based on partial rowkeys To: gsangudi@23andme.com Cc: dev@phoenix.apache.org Content-Type: multipart/alternative; boundary="000000000000eba5b60575c342d1" --000000000000eba5b60575c342d1 Content-Type: text/plain; charset="UTF-8" Gerald, I think you missed Josh's reply here : https://lists.apache.org/thread.html/c5145461805429622a410c23c1199d578e146a5c94511b2d5833438b@%3Cdev.phoenix.apache.org%3E Could you explain how using a subset of the pk columns to generate the salt byte helps with partitioning, aggregations etc? Thanks, Thomas On Thu, Sep 13, 2018 at 8:32 AM, Gerald Sangudi wrote: > Hi folks, > > Any thoughts or feedback on this? > > Thanks, > Gerald > > On Mon, Sep 10, 2018 at 1:56 PM, Gerald Sangudi > wrote: > >> Hello folks, >> >> We have a requirement for salting based on partial, rather than full, >> rowkeys. My colleague Mike Polcari has identified the requirement and >> proposed an approach. >> >> I found an already-open JIRA ticket for the same issue: >> https://issues.apache.org/jira/browse/PHOENIX-4757. I can provide more >> details from the proposal. >> >> The JIRA proposes a syntax of SALT_BUCKETS(col, ...) = N, whereas Mike >> proposes SALT_COLUMN=col or SALT_COLUMNS=col, ... . >> >> The benefit at issue is that users gain more control over partitioning, >> and this can be used to push some additional aggregations and hash joins >> down to region servers. >> >> I would appreciate any go-ahead / thoughts / guidance / objections / >> feedback. I'd like to be sure that the concept at least is not >> objectionable. We would like to work on this and submit a patch down the >> road. I'll also add a note to the JIRA ticket. >> >> Thanks, >> Gerald >> >> > --000000000000eba5b60575c342d1--