hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anubhav Tarar <anubhav.ta...@knoldus.in>
Subject Re: Unable to Use Vectorized Reader In Hive
Date Fri, 14 Jul 2017 05:52:01 GMT
so should it be update in hive cwiki that When Fetch is used in the plan
instead of Map, it do not vectorize.

On Thu, Jul 13, 2017 at 6:59 PM, Matthew McCline <mmccline@hortonworks.com>
wrote:

>
> When Fetch is used in the plan instead of Map, we do not vectorize.
>
> Suggest using:
> set hive.fetch.task.conversion=none;
>
> ________________________________________
> From: Anubhav Tarar <anubhav.tarar@knoldus.in>
> Sent: Thursday, July 13, 2017 6:03 AM
> To: dev@hive.apache.org
> Subject: Unable to Use Vectorized Reader In Hive
>
> hi, i am uanble to use vectorized reader with hive 1.2.1 i use this link
> from hive documentation
>
> https://cwiki.apache.org/confluence/display/Hive/
> Vectorized+Query+Execution
>
> step1: create table check(id int) stored as ORC;
> OK
> Time taken: 0.521 seconds
>
> step2:hive> insert into check values(1);
> Query ID = hduser_20170713162335_9bb368e6-c11f-4d87-8edf-86484fb0a233
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Job running in-process (local Hadoop)
> 2017-07-13 16:23:39,323 Stage-1 map = 0%,  reduce = 0%
> 2017-07-13 16:23:40,390 Stage-1 map = 100%,  reduce = 0%
> Ended Job = job_local800495047_0001
> Stage-4 is selected by condition resolver.
> Stage-3 is filtered out by condition resolver.
> Stage-5 is filtered out by condition resolver.
> Moving data to:
> hdfs://localhost:54310/usr/hive/warehouse/check/.hive-
> staging_hive_2017-07-13_16-23-35_493_7294075038374724529-1/-ext-10000
> Loading data to table default.check
> Table default.check stats: [numFiles=1, numRows=1, totalSize=199,
> rawDataSize=4]
> MapReduce Jobs Launched:
> Stage-Stage-1:  HDFS Read: 4055 HDFS Write: 270 SUCCESS
> Total MapReduce CPU Time Spent: 0 msec
> OK
> Time taken: 5.505 seconds
> step3:
> hive> explain select * from check where id >1;
> OK
> STAGE DEPENDENCIES:
>  Stage-0 is a root stage
>
> STAGE PLANS:
>  Stage: Stage-0
>    Fetch Operator
>      limit: -1
>      Processor Tree:
>        TableScan
>          alias: check
>          Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column
> stats: NONE
>          Filter Operator
>            predicate: (id > 1) (type: boolean)
>            Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE
> Column stats: NONE
>            Select Operator
>              expressions: 1 (type: int)
>              outputColumnNames: _col0
>              Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE
> Column stats: NONE
>
>
> explain command does not show that vectorized reader is applied
> --
> Thanks and Regards
>
> *   Anubhav Tarar     *
>
>
> * Software Consultant*
>       *Knoldus Software LLP <http://www.knoldus.com/home.knol>       *
>        LinkedIn <http://in.linkedin.com/in/rahulforallp>     Twitter
> <https://twitter.com/RahulKu71223673>    fb <rahulforallp@facebook.com>
>           mob : 8588915184
>
>
>


-- 
Thanks and Regards

*   Anubhav Tarar     *


* Software Consultant*
      *Knoldus Software LLP <http://www.knoldus.com/home.knol>       *
       LinkedIn <http://in.linkedin.com/in/rahulforallp>     Twitter
<https://twitter.com/RahulKu71223673>    fb <rahulforallp@facebook.com>
          mob : 8588915184

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message