hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <jornfra...@gmail.com>
Subject Re: Vectorised Query Execution extension
Date Fri, 05 Aug 2016 05:55:46 GMT
Even if it is possible it does only make sense to a certain limit given by your CPU and CPU
caches.

> On 04 Aug 2016, at 22:57, Mich Talebzadeh <mich.talebzadeh@gmail.com> wrote:
> 
> As I understand from the manual:
> 
> Vectorized query execution is a Hive feature that greatly reduces the CPU usage for typical
query operations like scans, filters, aggregates, and joins. A standard query execution system
processes one row at a time. This involves long code ...... Vectorized query execution streamlines
operations by processing a block of 1024 rows at a time. Within the block, each column is
stored as a vector (an array of a primitive data type).
> 
> As fart as I can see Vectorized query execution (VQE) can be applied to most columns
and sql operations. Is it therefore possible to extend it beyond 1024 rows to include the
whole column in table?
> 
> VQE would be very useful especially with ORC as it basically means that one can process
the whole column separately thus improving performance of the query.
> 
> HTH
> 
> Dr Mich Talebzadeh
>  
> LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>  
> http://talebzadehmich.wordpress.com
> 
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage
or destruction of data or any other property which may arise from relying on this email's
technical content is explicitly disclaimed. The author will in no case be liable for any monetary
damages arising from such loss, damage or destruction.
>  

Mime
View raw message