Return-Path: X-Original-To: apmail-hive-commits-archive@www.apache.org Delivered-To: apmail-hive-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3C8C918B08 for ; Thu, 3 Dec 2015 19:45:44 +0000 (UTC) Received: (qmail 38826 invoked by uid 500); 3 Dec 2015 19:45:44 -0000 Delivered-To: apmail-hive-commits-archive@hive.apache.org Received: (qmail 38780 invoked by uid 500); 3 Dec 2015 19:45:44 -0000 Mailing-List: contact commits-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hive-dev@hive.apache.org Delivered-To: mailing list commits@hive.apache.org Received: (qmail 38758 invoked by uid 99); 3 Dec 2015 19:45:44 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Dec 2015 19:45:44 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id CBCCCE6827; Thu, 3 Dec 2015 19:45:43 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sershe@apache.org To: commits@hive.apache.org Date: Thu, 03 Dec 2015 19:45:43 -0000 Message-Id: <142c7ca624d24354be8a1820de9bb1d2@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/2] hive git commit: HIVE-12532 : LLAP Cache: Uncompressed data cache has NPE (Sergey Shelukhin, reviewed by Prasanth Jayachandran) Repository: hive Updated Branches: refs/heads/branch-2.0 4c8e47eee -> 1946ccbcd refs/heads/master a603ed8d7 -> 1d02ab578 HIVE-12532 : LLAP Cache: Uncompressed data cache has NPE (Sergey Shelukhin, reviewed by Prasanth Jayachandran) Project: http://git-wip-us.apache.org/repos/asf/hive/repo Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/1d02ab57 Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/1d02ab57 Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/1d02ab57 Branch: refs/heads/master Commit: 1d02ab578dbd47103a70710abd4d949ea8cea9d2 Parents: a603ed8 Author: Sergey Shelukhin Authored: Thu Dec 3 11:44:10 2015 -0800 Committer: Sergey Shelukhin Committed: Thu Dec 3 11:44:10 2015 -0800 ---------------------------------------------------------------------- .../hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hive/blob/1d02ab57/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java ---------------------------------------------------------------------- diff --git a/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java b/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java index f789a4f..a8b51b9 100644 --- a/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java +++ b/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java @@ -778,19 +778,18 @@ class EncodedReaderImpl implements EncodedReader { } // Account for maximum cache buffer size. long streamLen = streamEnd - streamOffset; - int partSize = determineUncompressedPartSize(), // + int partSize = determineUncompressedPartSize(), partCount = (int)(streamLen / partSize) + (((streamLen % partSize) != 0) ? 1 : 0); CacheChunk lastUncompressed = null; MemoryBuffer[] singleAlloc = new MemoryBuffer[1]; - /* -Starting pre-read for [12187411,17107411) at start: 12187411 end: 12449555 cache buffer: 0x5f64a8f6(2) -Processing uncompressed file data at [12187411, 12449555) - */ for (int i = 0; i < partCount; ++i) { long partOffset = streamOffset + (i * partSize), partEnd = Math.min(partOffset + partSize, streamEnd); long hasEntirePartTo = partOffset; // We have 0 bytes of data for this part, for now. + if (current == null) { + break; // We have no data from this point on (could be unneeded), skip. + } assert partOffset <= current.getOffset(); if (partOffset == current.getOffset() && current instanceof CacheChunk) { // We assume cache chunks would always match the way we read, so check and skip it.