Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Fri, 3 May 2013 18:42:16 +0000 (UTC)
From: "Carl Steinbach (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12617683.1353963583426.266696.1367606536614@arcas>
In-Reply-To: <JIRA.12617683.1353963583426@arcas>
References: <JIRA.12617683.1353963583426@arcas>
Subject: [jira] [Commented] (HIVE-3746) TRowSet resultset structure should
 be column-oriented
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648667#comment-13648667 ] 

Carl Steinbach commented on HIVE-3746:
--------------------------------------

bq. If an application has requested a single row, and the client has requested n rows from the server in an effort to reduce round trips, then n-1 intervening values from the first column must be cached off somewhere before the first value for the second column can be accessed.

If the fetch size is n, then the client is going to end up storing n rows in memory regardless of whether the result set is represented in a row-major or column-major format. Put another way, the unit of data transfer between the server and client is a variable sized resultset. The client has the option of setting the result size very low in order to achieve lower latency, or making it very large in order to get higher overall throughput. However, the key limitation is that the client is not able to provide access to any of the rows contained in a resultset until the entire resultset has been transferred from the server to the client. This limitation is a consequence of the fact that we're using a message oriented RPC layer (Thrift) to handle communication and data transfer between the client and server.
                
> TRowSet resultset structure should be column-oriented
> -----------------------------------------------------
>
>                 Key: HIVE-3746
>                 URL: https://issues.apache.org/jira/browse/HIVE-3746
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Server Infrastructure
>            Reporter: Carl Steinbach
>            Assignee: Carl Steinbach
>              Labels: HiveServer2
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira