hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin" <ser...@hortonworks.com>
Subject Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead
Date Fri, 09 May 2014 20:10:19 GMT


> On May 9, 2014, 1:58 a.m., Gunther Hagleitner wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/io/DateWritable.java, line 147
> > <https://reviews.apache.org/r/18936/diff/13/?file=572150#file572150line147>
> >
> >     randomaccess doesn't extend output?

no


- Sergey


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18936/#review42555
-----------------------------------------------------------


On May 1, 2014, 2:29 a.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18936/
> -----------------------------------------------------------
> 
> (Updated May 1, 2014, 2:29 a.m.)
> 
> 
> Review request for hive, Gopal V and Gunther Hagleitner.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> See JIRA
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 604bea7 
>   conf/hive-default.xml.template 2552560 
>   hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 5fe35a5 
>   hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
142bfd8 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java bf9d4c1 
>   ql/src/java/org/apache/hadoop/hive/ql/debug/Utils.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java f5d4670 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java b93ea7a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 175d3ab 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/AbstractMapJoinTableContainer.java
8854b19 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 9df425b

>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 64f0be2 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinPersistableTableContainer.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinRowContainer.java 008a8db

>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java 988959f

>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainerSerDe.java
55b7415 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java e392592 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java eef7656

>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedColumnarSerDe.java d4be78d

>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 3077d75 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java f7b499b

>   ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 157d072 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 118b339 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestBytesBytesMultiHashMap.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinEqualityTableContainer.java
65e3779 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinTableContainer.java
093da55 
>   ql/src/test/queries/clientpositive/mapjoin_decimal.q b65a7be 
>   ql/src/test/queries/clientpositive/mapjoin_mapjoin.q 1eb95f6 
>   ql/src/test/queries/clientpositive/tez_union.q f80d94c 
>   ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out 8350670 
>   ql/src/test/results/clientpositive/tez/mapjoin_decimal.q.out 3c55b5c 
>   ql/src/test/results/clientpositive/tez/mapjoin_mapjoin.q.out 284cc03 
>   serde/src/java/org/apache/hadoop/hive/serde2/ByteStream.java 73d9b29 
>   serde/src/java/org/apache/hadoop/hive/serde2/WriteBuffers.java PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
9079b9d 
>   serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/OutputByteBuffer.java 1b09d41

>   serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java 5870884 
>   serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java
bab505e 
>   serde/src/java/org/apache/hadoop/hive/serde2/dynamic_type/DynamicSerDe.java 6f344bb

>   serde/src/java/org/apache/hadoop/hive/serde2/io/DateWritable.java 1f4ccdd 
>   serde/src/java/org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.java a99c7b4 
>   serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java 435d6c6 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java 82c1263 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java b188c3f

>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java caf3517

>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 6c14081

>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryStructObjectInspector.java
e5ea452 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java
06d5c5e 
>   serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 868dd4c 
>   serde/src/test/org/apache/hadoop/hive/serde2/thrift_test/CreateSequenceFile.java 1fb49e5

> 
> Diff: https://reviews.apache.org/r/18936/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message