hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jitendra Pandey <jiten...@hortonworks.com>
Subject Re: Vectorizied execution on RCFile
Date Fri, 10 Jan 2014 18:35:15 GMT
Hi Rajesh,
  Some parts of the patch on HIVE-4483 are already committed to the trunk.
So the patch may not cleanly apply. Once you have this patch, you would
need to use CommonRCFileInputFormat (that this patch provides). We haven't
included this input-format in the trunk because we intend to enhance
regular RCFileInputFormat itself to provide both vector and non-vector data
in future. Another caveat is that vectorized reader for rc file still reads
row by row and populates vectorized batches, which is sub-optimal,
therefore you won't see as much performance gain as ORC, as Eric pointed
out.

jitendra


On Wed, Jan 8, 2014 at 6:46 PM, Rajesh Balamohan <rajesh.balamohan@gmail.com
> wrote:

> Hi All,
>
> Vectorization with ORCFile provides amazing performance.  Does
> vectorization work with RCFile as well?
>
> As per explain plan of Hive 0.13 (snapshot), it does not use vectorization
> with RCFile.  Any pointers would be appreciated.
>
>
> --
> ~Rajesh.B
>



-- 
<http://hortonworks.com/download/>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message