hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sahil Takiar <takiar.sa...@gmail.com>
Subject Re: Review Request 49782: HIVE-14170: Beeline IncrementalRows should buffer rows and incrementally re-calculate width if TableOutputFormat is used
Date Mon, 11 Jul 2016 17:29:48 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49782/
-----------------------------------------------------------

(Updated July 11, 2016, 5:29 p.m.)


Review request for hive, Sergio Pena, Thejas Nair, and Vaibhav Gumashta.


Changes
-------

Addressing comments


Bugs: HIVE-14170
    https://issues.apache.org/jira/browse/HIVE-14170


Repository: hive-git


Description
-------

* Added a new BeeLine Options called `incrementalBufferRows` which controls the number of
`Row`s the `IncrementalRows` class should buffer, by default it is 1000
* Modified `BufferedRows` so that it can accept a limit on the number of `Row`s it buffers
* Modified `IncrementalRows` to read the value of `incrementalBufferRows` and buffer rows
as per HIVE-14170
* The class delegates all buffering work to a `BufferedRows` class
* This has the advantage that all the width calculaltion that spans multiple rows can be encapsulate
in the `BufferedRows` class, there is no need to re-implement the logic in `IncrementalRows`
* `IncrementalRows` will buffer `incrementalBufferRows` rows at a time, when the buffer is
depleted, it will fetch the next buffer and re-calculate the width for that buffer


Diffs (updated)
-----

  beeline/pom.xml a720d08 
  beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 5aaa385 
  beeline/src/java/org/apache/hive/beeline/BufferedRows.java 962c531 
  beeline/src/java/org/apache/hive/beeline/IncrementalRows.java 8aef976 
  beeline/src/java/org/apache/hive/beeline/Rows.java 453f685 
  beeline/src/main/resources/BeeLine.properties 7500df9 
  beeline/src/test/org/apache/hive/beeline/TestIncrementalRows.java PRE-CREATION 

Diff: https://reviews.apache.org/r/49782/diff/


Testing
-------

* Unit Test added for `IncrementalRows`
* Tested locally


Thanks,

Sahil Takiar


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message