Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 30 Sep 2015 22:40:06 +0000 (UTC)
From: "Ashutosh Chauhan (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.12772211.1423010830000.126639.1443652806048@Atlassian.JIRA>
In-Reply-To: <JIRA.12772211.1423010830000@Atlassian.JIRA>
References: <JIRA.12772211.1423010830000@Atlassian.JIRA>
 <JIRA.12772211.1423010830377@arcas>
Subject: [jira] [Commented] (HIVE-9573) Lazy load partitions for SELECT
 LIMIT type query
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938990#comment-14938990 ] 

Ashutosh Chauhan commented on HIVE-9573:
----------------------------------------

This indeed sounds interesting. Will you like to upload a patch?

> Lazy load partitions for SELECT LIMIT type query
> ------------------------------------------------
>
>                 Key: HIVE-9573
>                 URL: https://issues.apache.org/jira/browse/HIVE-9573
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Selina Zhang
>            Assignee: Selina Zhang
>
> Some tools such as HUE uses 
> SELECT * FROM table LIMIT 100;
> to grab the sample content of table. For table with large partitions it causes a large amount of partition objects been loaded. Thus slow down the HS2 even cause OOM from time to time. 
> My solution is lazy load partition objects in FetchOperator for this type of query. We can skip retrieving the whole partitions but only partition names in PartitionPrunner, and lazy load the partition objects when needed in execution time (for local job only).
> I have a patch ready. But want to hear more suggestions. Thanks!


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)