Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0B39110BB3 for ; Fri, 7 Mar 2014 22:12:28 +0000 (UTC) Received: (qmail 61268 invoked by uid 500); 7 Mar 2014 22:12:25 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 61210 invoked by uid 500); 7 Mar 2014 22:12:25 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 61198 invoked by uid 99); 7 Mar 2014 22:12:25 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Mar 2014 22:12:25 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 66E831D4C79; Fri, 7 Mar 2014 22:12:23 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============8032611008460518531==" MIME-Version: 1.0 Subject: Review Request 18925: HIVE-6575 select * fails on parquet table with map datatype From: "Szehon Ho" To: "justin coffey" , "Xuefu Zhang" , "Brock Noland" Cc: "Szehon Ho" , "hive" Date: Fri, 07 Mar 2014 22:12:23 -0000 Message-ID: <20140307221223.2625.17273@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Szehon Ho" X-ReviewGroup: hive X-ReviewRequest-URL: https://reviews.apache.org/r/18925/ X-Sender: "Szehon Ho" Reply-To: "Szehon Ho" X-ReviewRequest-Repository: hive-git --===============8032611008460518531== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18925/ ----------------------------------------------------------- Review request for hive, Brock Noland, justin coffey, and Xuefu Zhang. Repository: hive-git Description ------- The issue is, as part of select * query, a DeepParquetHiveMapInspector is used for one column of an overall parquet-table struct object inspector. The problem lies in the ObjectInspectorFactory's cache for struct object inspector. For performance it caches based on an array list, of all object inspectors of columns. The second time the query is run, it is attempted to lookup cached value. When the DeepParquetHiveMapInspector is gotten to, java calls .equals but it fails as the .equals method casted it to a "StandardParquetHiveInspector". Regenerating the .equals and .hashcode from eclipse. Also adding one more check in .equals before casting, as if by luck another type of object inspector gets hashed to the same hashcode in the cache, java would call .equals against it to find it if it is the key. Diffs ----- ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/AbstractParquetMapInspector.java 1d72747 Diff: https://reviews.apache.org/r/18925/diff/ Testing ------- Manual testing. Thanks, Szehon Ho --===============8032611008460518531==--