Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 67083200C14 for ; Tue, 7 Feb 2017 22:29:09 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 65A16160B69; Tue, 7 Feb 2017 21:29:09 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id AD50F160B68 for ; Tue, 7 Feb 2017 22:29:08 +0100 (CET) Received: (qmail 29829 invoked by uid 500); 7 Feb 2017 21:29:07 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 29763 invoked by uid 99); 7 Feb 2017 21:29:07 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Feb 2017 21:29:07 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id C153C2DF8A7; Tue, 7 Feb 2017 21:29:06 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============5765820151595851112==" MIME-Version: 1.0 Subject: Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II From: Owen O'Malley To: Gopal V , j.prasanth.j@gmail.com Cc: Owen O'Malley , Sergey Shelukhin , hive Date: Tue, 07 Feb 2017 21:29:06 -0000 Message-ID: <20170207212906.31774.61720@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: Owen O'Malley X-ReviewGroup: hive X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/55977/ X-Sender: Owen O'Malley References: <20170207023152.31774.10934@reviews.apache.org> In-Reply-To: <20170207023152.31774.10934@reviews.apache.org> Reply-To: Owen O'Malley X-ReviewRequest-Repository: hive-git archived-at: Tue, 07 Feb 2017 21:29:09 -0000 --===============5765820151595851112== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit > On Feb. 7, 2017, 2:31 a.m., Owen O'Malley wrote: > > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 360 > > > > > > You need braces around your control block for break. > > > > Actually, this is an error to have an include vector that is shorter that the reader type and should cause a thrown exception. Although it should be a single check that include.length == readerSchema.getMaxId() + 1. > > Sergey Shelukhin wrote: > as far as I remember, I've seen vectorization(?) use trimmed includes somewhere > > Owen O'Malley wrote: > Nope. The include vector has to exactly match the reader schema, although it doesn't include the ACID columns. In fact, the current version of SchemaEvolution in ORC will throw in such cases. > > Sergey Shelukhin wrote: > See e.g. VectorMapOperator - determineDataColumnsToIncludeTruncated If that boolean array is passed down to an ORC reader (or directly to SchemaEvolution), it will fail by design. - Owen ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55977/#review164437 ----------------------------------------------------------- On Feb. 7, 2017, 6:35 p.m., Sergey Shelukhin wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55977/ > ----------------------------------------------------------- > > (Updated Feb. 7, 2017, 6:35 p.m.) > > > Review request for hive, Gopal V and Prasanth_J. > > > Repository: hive-git > > > Description > ------- > > see jira > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cb27cd6 > llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java 85fae9a > llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java 4295c1c > llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java 8d86d17 > llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java 63a3be2 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java a434763 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 > storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java b894c11e > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java bbd9ca6 > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 6f090a1 > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java e4f8d82 > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java bd421f4 > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java c4a6c0f > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java 80d4731 > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java 1aeff83 > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java cf07bca > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java 28997a0 > storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java 0c61243 > > Diff: https://reviews.apache.org/r/55977/diff/ > > > Testing > ------- > > > Thanks, > > Sergey Shelukhin > > --===============5765820151595851112==--