Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B36AE11E18 for ; Tue, 2 Sep 2014 13:32:21 +0000 (UTC) Received: (qmail 63927 invoked by uid 500); 2 Sep 2014 13:32:21 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 63853 invoked by uid 500); 2 Sep 2014 13:32:21 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 63839 invoked by uid 500); 2 Sep 2014 13:32:21 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 63836 invoked by uid 99); 2 Sep 2014 13:32:21 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Sep 2014 13:32:21 +0000 Date: Tue, 2 Sep 2014 13:32:21 +0000 (UTC) From: "Hive QA (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-7832) Do ORC dictionary check at a finer level and preserve encoding across stripes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118192#comment-14118192 ] Hive QA commented on HIVE-7832: ------------------------------- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12665866/HIVE-7832.9.patch {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 6141 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/602/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/602/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-602/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12665866 > Do ORC dictionary check at a finer level and preserve encoding across stripes > ----------------------------------------------------------------------------- > > Key: HIVE-7832 > URL: https://issues.apache.org/jira/browse/HIVE-7832 > Project: Hive > Issue Type: Improvement > Affects Versions: 0.14.0 > Reporter: Prasanth J > Assignee: Prasanth J > Attachments: HIVE-7832.1.patch, HIVE-7832.2.patch, HIVE-7832.3.patch, HIVE-7832.4.patch, HIVE-7832.5.patch, HIVE-7832.6.patch, HIVE-7832.7.patch, HIVE-7832.8.patch, HIVE-7832.9.patch > > > Currently ORC dictionary check happens while writing the stripe. Just before writing stripe if ratio of dictionary entries to total non-null rows is greater than threshold then the dictionary is discarded. Also, the decision of using dictionary or not is preserved across stripes. This sometimes leads to costly insertion cost of O(logn) for each stripes when there are too many distinct keys. -- This message was sent by Atlassian JIRA (v6.3.4#6332)