hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II
Date Tue, 07 Feb 2017 21:29:06 GMT


> On Feb. 7, 2017, 2:31 a.m., Owen O'Malley wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 360
> > <https://reviews.apache.org/r/55977/diff/6/?file=1625756#file1625756line360>
> >
> >     You need braces around your control block for break.
> >     
> >     Actually, this is an error to have an include vector that is shorter that the
reader type and should cause a thrown exception. Although it should be a single check that
include.length == readerSchema.getMaxId() + 1.
> 
> Sergey Shelukhin wrote:
>     as far as I remember, I've seen vectorization(?) use trimmed includes somewhere
> 
> Owen O'Malley wrote:
>     Nope. The include vector has to exactly match the reader schema, although it doesn't
include the ACID columns. In fact, the current version of SchemaEvolution in ORC will throw
in such cases.
> 
> Sergey Shelukhin wrote:
>     See e.g. VectorMapOperator - determineDataColumnsToIncludeTruncated

If that boolean array is passed down to an ORC reader (or directly to SchemaEvolution), it
will fail by design.


- Owen


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review164437
-----------------------------------------------------------


On Feb. 7, 2017, 6:35 p.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> -----------------------------------------------------------
> 
> (Updated Feb. 7, 2017, 6:35 p.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cb27cd6 
>   llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
85fae9a 
>   llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
4295c1c 
>   llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
8d86d17 
>   llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
63a3be2 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
a434763 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
b894c11e 
>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java bbd9ca6

>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 6f090a1

>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
e4f8d82 
>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
bd421f4 
>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
c4a6c0f 
>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java 80d4731

>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
1aeff83 
>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
cf07bca 
>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
28997a0 
>   storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java 0c61243

> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message