hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-3234) getting the reporter in the recordwriter
Date Tue, 27 Nov 2012 00:22:58 GMT

     [ https://issues.apache.org/jira/browse/HIVE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HIVE-3234:
------------------------------

    Attachment: HIVE-3234.D6987.1.patch

omalley requested code review of "HIVE-3234 [jira] getting the reporter in the recordwriter".
Reviewers: JIRA

  HIVE-3736 : hive unit test case build failure. (Ashish Singh via Ashutosh Chauhan)

  We would like to generate some custom statistics and report back to map/reduce later wen
implement the
   FileSinkOperator.RecordWriter interface. However, the current interface design doesn't
allow us to get the map reduce reporter object. Please extend the current FileSinkOperator.RecordWriter
interface so that it's close() method passes in a map reduce reporter object.

  For the same reason, please also extend the RecordReader interface too to include a reporter
object so that users can passes in custom map reduce  counters.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D6987

AFFECTED FILES
  ivy/ivysettings.xml
  ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/JoinUtil.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/RowContainer.java
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java
  ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java
  ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileMergeMapper.java
  ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13InputFormat.java
  ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13OutputFormat.java
  ql/src/test/queries/clientpositive/custom_input_output_format.q
  ql/src/test/results/clientpositive/custom_input_output_format.q.out

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/16461/

To: JIRA, omalley

                
> getting the reporter in the recordwriter
> ----------------------------------------
>
>                 Key: HIVE-3234
>                 URL: https://issues.apache.org/jira/browse/HIVE-3234
>             Project: Hive
>          Issue Type: Improvement
>          Components: Serializers/Deserializers
>    Affects Versions: 0.9.1
>         Environment: any
>            Reporter: Jimmy Hu
>            Assignee: Owen O'Malley
>              Labels: newbie
>             Fix For: 0.9.1
>
>         Attachments: HIVE-3234.D6699.1.patch, HIVE-3234.D6699.2.patch, HIVE-3234.D6987.1.patch
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> We would like to generate some custom statistics and report back to map/reduce later
wen implement the 
>  FileSinkOperator.RecordWriter interface. However, the current interface design doesn't
allow us to get the map reduce reporter object. Please extend the current FileSinkOperator.RecordWriter
interface so that it's close() method passes in a map reduce reporter object. 
> For the same reason, please also extend the RecordReader interface too to include a reporter
object so that users can passes in custom map reduce  counters.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message