hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harish Butani" <rhbut...@gmail.com>
Subject Re: Review Request 21168: HIVE-6999: Add streaming mode to PTFs
Date Wed, 14 May 2014 21:21:27 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21168/
-----------------------------------------------------------

(Updated May 14, 2014, 9:21 p.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-6999
    https://issues.apache.org/jira/browse/HIVE-6999


Repository: hive-git


Description
-------

There are a set of use cases where the Table Function can operate on a Partition row by row
or on a subset(window) of rows as it is being streamed to it.
Windowing has couple of use cases of this:processing of Rank functions, processing of Window
Aggregations.
But this is a generic concept: any analysis that operates on an Ordered partition maybe able
to operate in Streaming mode.
This patch introduces streaming mode in PTFs and provides the mechanics to handle PTF chains
that contain both modes of PTFs.
Subsequent patches will introduce Streaming mode for Windowing.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 3bb8fa9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/PTFOperator.java 4d314b7 
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java 34aebf0 
  ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/NoopStreaming.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/NoopWithMapStreaming.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/TableFunctionEvaluator.java 1087bbf 
  ql/src/test/queries/clientpositive/ptf_streaming.q PRE-CREATION 
  ql/src/test/results/clientpositive/ptf_streaming.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/21168/diff/


Testing
-------

added new tests


Thanks,

Harish Butani


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message