hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (JIRA)" <>
Subject [jira] [Commented] (HIVE-3746) TRowSet resultset structure should be column-oriented
Date Fri, 03 May 2013 18:42:16 GMT


Carl Steinbach commented on HIVE-3746:

bq. If an application has requested a single row, and the client has requested n rows from
the server in an effort to reduce round trips, then n-1 intervening values from the first
column must be cached off somewhere before the first value for the second column can be accessed.

If the fetch size is n, then the client is going to end up storing n rows in memory regardless
of whether the result set is represented in a row-major or column-major format. Put another
way, the unit of data transfer between the server and client is a variable sized resultset.
The client has the option of setting the result size very low in order to achieve lower latency,
or making it very large in order to get higher overall throughput. However, the key limitation
is that the client is not able to provide access to any of the rows contained in a resultset
until the entire resultset has been transferred from the server to the client. This limitation
is a consequence of the fact that we're using a message oriented RPC layer (Thrift) to handle
communication and data transfer between the client and server.
> TRowSet resultset structure should be column-oriented
> -----------------------------------------------------
>                 Key: HIVE-3746
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Server Infrastructure
>            Reporter: Carl Steinbach
>            Assignee: Carl Steinbach
>              Labels: HiveServer2

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message