Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C583E200BA6 for ; Tue, 4 Oct 2016 01:45:13 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id C3880160AE5; Mon, 3 Oct 2016 23:45:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0B85B160ADC for ; Tue, 4 Oct 2016 01:45:12 +0200 (CEST) Received: (qmail 36689 invoked by uid 500); 3 Oct 2016 23:45:11 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 36679 invoked by uid 99); 3 Oct 2016 23:45:11 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Oct 2016 23:45:11 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 1C580C0185 for ; Mon, 3 Oct 2016 23:45:11 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.979 X-Spam-Level: * X-Spam-Status: No, score=1.979 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=xpldr-com.20150623.gappssmtp.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id dxW8D0ThEHwO for ; Mon, 3 Oct 2016 23:45:10 +0000 (UTC) Received: from mail-wm0-f46.google.com (mail-wm0-f46.google.com [74.125.82.46]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id D41FD5F253 for ; Mon, 3 Oct 2016 23:45:09 +0000 (UTC) Received: by mail-wm0-f46.google.com with SMTP id b201so111526116wmb.0 for ; Mon, 03 Oct 2016 16:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xpldr-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=5u1bqLiboqSuWVd414yrjyr+M7MAlc6rNzvRvXjJ00Q=; b=ia9//xwVMOgF4pO+vkt+Nzs87HmSy0nfi11+X03Aosz+9Np7dWLKq/p0Lc0tcO+tya udn+OF9HIKBYcOwnt4rV+vou9EoNkdz7Aukw2WNm2U1nQrrgrBlR/QFybpTi1CqhjVPX WSpMbNm/KkWs14gfAHnamwKITYZFp9q70kkvWxeTAehXYh6pOkBZaIYuJeJSQNIujk0I U/EFTy7EEif2tm50h2DQ4cziakp6WzTVs7DVI9SPPxtDdq+Ccpjys/LtmSr+yM6F5Uff ApBzQYIefJTTQiwXuekXasr8vOSaj9jEvSIPWrZnw9U/e/7aWW/k0i1/7csEEe9BeXxP 1B8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=5u1bqLiboqSuWVd414yrjyr+M7MAlc6rNzvRvXjJ00Q=; b=T6J776zTZThaKxr9EaHoab/G1iDIvWo9jpScPMpBOYE4/4egqSuSLnyz2wq5pgdGec 0JPgsMLh5Eb/wb1eOOIvuuAWETK/prM6YxJ13TBLDss9pCEaEBeR6l8lnJp/zJDkCV+a RYWo5mRgYJPVJnFRvHpzRAdK5SUra1QvpE0GTn9POpsXLbCL7/KNZd+/iaYXSnGTfR1q N5zgZ5A1/qEpWx2MDLaUa4/uCJc1llyQ712ybT4Tlfqht0NS/L6IxU1BafVwlkqYjmmC kB7wxZh/NqFMZVwP01znhmQzLHewO9rN7v0RSyrzXe+XsodXWv2CKWk3HU7MQ49RYLtp Wj7g== X-Gm-Message-State: AA6/9RkQqhn8GXfBx73LugHl1+95kYcWgB8m1pLa25Vg3j7KB8g2r6bR15u3bNODlScOtbo3HfttZJTPtzvHlw== X-Received: by 10.194.150.206 with SMTP id uk14mr384014wjb.139.1475538308803; Mon, 03 Oct 2016 16:45:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.44.8 with HTTP; Mon, 3 Oct 2016 16:45:08 -0700 (PDT) From: Owen Thomas Date: Mon, 3 Oct 2016 16:45:08 -0700 Message-ID: Subject: StorageHandler and Hive Cost Based Optimization To: user@hive.apache.org Content-Type: multipart/alternative; boundary=089e01228c98bfc416053dfe88b4 archived-at: Mon, 03 Oct 2016 23:45:13 -0000 --089e01228c98bfc416053dfe88b4 Content-Type: text/plain; charset=UTF-8 Hi, I'd like to know if its possible for a StorageHandler backed table to contribute more than table-wide statistics during CBO analysis. Specifically, with native partitioned tables, CBO will filter partitions not involved in the query. Is it possible to do something similar to the StorageHandler's predicate push-down optimization for CBO? Ideally, I'd like an interface that lets me generate a Statistics object with respect to a Table and given predicate. This mirrors the HiveStoragePredicateHandler, but for statistics generation. Is there an approach for doing this or do I need to fall back to native tables and partitions? Apologies if this question has been asked before or this isn't the correct audience. Regards, Owen. --089e01228c98bfc416053dfe88b4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,

I'd like to know if its possibl= e for a StorageHandler backed table to contribute more than table-wide stat= istics during CBO analysis. Specifically, with native partitioned tables, C= BO will filter partitions not involved in the query. Is it possible to do s= omething similar to the StorageHandler's predicate push-down optimizati= on for CBO?

Ideally, I'd like an interface tha= t lets me generate a Statistics object with respect to a Table and given pr= edicate. This mirrors the=C2=A0HiveStoragePredicateHandler, but for statist= ics generation.

Is there an approach for doing thi= s or do I need to fall back to native tables and partitions?=C2=A0

Apologies if this question has been asked before or this i= sn't the correct audience.

Regards,
=
Owen.

--089e01228c98bfc416053dfe88b4--