From user-return-733-archive-asf-public=cust-asf.ponee.io@impala.apache.org Tue Mar 20 08:45:23 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9118218064A for ; Tue, 20 Mar 2018 08:45:22 +0100 (CET) Received: (qmail 80761 invoked by uid 500); 20 Mar 2018 07:45:21 -0000 Mailing-List: contact user-help@impala.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@impala.apache.org Delivered-To: mailing list user@impala.apache.org Received: (qmail 80751 invoked by uid 99); 20 Mar 2018 07:45:21 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Mar 2018 07:45:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id DC04A1A0591 for ; Tue, 20 Mar 2018 07:45:20 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.984 X-Spam-Level: *** X-Spam-Status: No, score=3.984 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FSL_BULK_SIG=0.001, HTML_MESSAGE=2, PYZOR_CHECK=1.985, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=onevmw.onmicrosoft.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 3bmeV67CIU2u for ; Tue, 20 Mar 2018 07:45:16 +0000 (UTC) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0079.outbound.protection.outlook.com [104.47.38.79]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id ECC2E5F1EC for ; Tue, 20 Mar 2018 07:45:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3vS7C6jr+mF6rJJ2A+ncbprp5nvzI7iWMX6mJUvazVA=; b=h+tpAqnssMJeuZZzYD4ivvjvxu+3OKCLE18SjyJ74oe1rHhOOYJfRoUZ6TekziKJCvXd8RqZeWY+wSEcy3POHUVZvdXByyFBkr0ZLdFUsmlu2xJAg5nt5WwjzQPwpsm9nxxQY4y5xC64rXi9gKRocCnzGYgBQQjn2fL79fJno24= Received: from SN1PR05MB1981.namprd05.prod.outlook.com (10.162.132.23) by SN1PR05MB2349.namprd05.prod.outlook.com (10.169.125.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.609.6; Tue, 20 Mar 2018 07:45:08 +0000 Received: from SN1PR05MB1981.namprd05.prod.outlook.com ([fe80::61d1:4c53:f02f:cb60]) by SN1PR05MB1981.namprd05.prod.outlook.com ([fe80::61d1:4c53:f02f:cb60%4]) with mapi id 15.20.0609.007; Tue, 20 Mar 2018 07:45:08 +0000 From: Antoni Ivanov To: "user@impala.apache.org" Subject: Does Impala supports or plan to support Late Materialization Thread-Topic: Does Impala supports or plan to support Late Materialization Thread-Index: AdPAHxo+uV1VWSXvR+C6lO64JeepvQ== Date: Tue, 20 Mar 2018 07:45:08 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [146.247.46.5] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;SN1PR05MB2349;7:8arkcKdntUHniUWJxj6mzhe8RWaj6erNazm+cD0MRujEBTEgfVtAqNwCaJ4b1z0SZdEu9ehL7p4lTSJ0/iT402t2eWZ643lwilO8kYmUbWSShqKqz17KqVKLKzwqKilM0RJl4WXlfzf0/k75CX/QMwKNgFdp7IQXxu9+aeuDYliwqf7V9ITO1XimImm8jG/f6vPF2Gg1rHjVK3UsnqWnv8AMtxUD21iHmm88oPGLFgp4iDdVmLkH6dRZsWz4txbR;20:6RZQrRspt02AX77RLq+jN1E3TOZoSi4KIVt/YDtEjYDbH1wvsTvtUOy/PoDxhRiMHfdvtlrHbU3NtpPK43/d3w+u/2MYo5zHYjWaUIjHVdNEgzGrS4odlXNLRNQPd52u4zXv2ps0J374D44IOiWQVkNrGHwT4UdSJ+GWfLFQIyk= x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 6954ccee-cbd6-45c3-cb36-08d58e368109 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020);SRVR:SN1PR05MB2349; x-ms-traffictypediagnostic: SN1PR05MB2349: authentication-results: spf=none (sender IP is ) smtp.mailfrom=aivanov@vmware.com; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(275133514327357)(28532068793085)(21748063052155)(47284530071512); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(10201501046)(3231221)(944501308)(52105095)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(6072148)(201708071742011);SRVR:SN1PR05MB2349;BCL:0;PCL:0;RULEID:;SRVR:SN1PR05MB2349; x-forefront-prvs: 061725F016 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(346002)(376002)(366004)(39860400002)(396003)(39380400002)(189003)(199004)(106356001)(74316002)(7736002)(25786009)(7696005)(1730700003)(81166006)(81156014)(8936002)(6916009)(8676002)(5660300001)(97736004)(53936002)(316002)(26005)(3660700001)(86362001)(6506007)(68736007)(3280700002)(186003)(66066001)(99286004)(33656002)(102836004)(105586002)(236005)(3846002)(6116002)(790700001)(606006)(5250100002)(2906002)(2501003)(2900100001)(14454004)(478600001)(2351001)(966005)(5640700003)(6436002)(55016002)(9686003)(6306002)(54896002);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR05MB2349;H:SN1PR05MB1981.namprd05.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; received-spf: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: vmjuFSIHJO0oXFhgMC+egaHLepNqqgUQT4T1SiCsORbQ69Pk+NorJGQUYNbIbOCHt1UQcrlYDPsMonPlwdm/8/YvJrkzB3km0cYxWISLeXm2tXCQG1f5MC04QqLn4TwQ8ed+aGkL58/wefEAmHbLcnsH05Gvm6Y9KNuFZZgg3ZfnDL/lw+aEb/7hbsjiqhviOdaSW6OCAeZvr0zEuLel3mHPl2IwsrZNC+gBuJ+PsfG0sRcApfz2Clv0cwKTsbAPb6cIEO89uksZYPF07nqySZbTGxD0msQkZgQN/t1cwhrtewan9BFSMJTLdL3zg2VDScLYSJ8SVOgBNfmSYU41lw== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_SN1PR05MB198144CF82194F7E6959217FC7AB0SN1PR05MB1981namp_" MIME-Version: 1.0 X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6954ccee-cbd6-45c3-cb36-08d58e368109 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Mar 2018 07:45:08.5723 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR05MB2349 --_000_SN1PR05MB198144CF82194F7E6959217FC7AB0SN1PR05MB1981namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I don't mean partition pruning but as described in https://aws.amazon.com/about-aws/whats-new/2017/12/amazon-redshift-introduc= es-late-materialization-for-faster-query-processing/ It basically pre-fetches first the filter columns and then after applying t= he filter it fetches only the data from the rest of columns only if filter = applies. Thanks --_000_SN1PR05MB198144CF82194F7E6959217FC7AB0SN1PR05MB1981namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I don’t mean partition pruning but as describe= d in

https://aws.amazon.com/about-aws/whats-new/2017/12/amazon-reds= hift-introduces-late-materialization-for-faster-query-processing/

 

It basically pre-fetches first the filter columns an= d then after applying the filter it fetches only the data from the rest of = columns only if filter applies.

 

Thanks

--_000_SN1PR05MB198144CF82194F7E6959217FC7AB0SN1PR05MB1981namp_--