hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <>
Subject [jira] [Commented] (HIVE-3234) getting the reporter in the recordwriter
Date Wed, 14 Nov 2012 18:18:13 GMT


Phabricator commented on HIVE-3234:

ashutoshc has requested changes to the revision "HIVE-3234 [jira] getting the reporter in
the recordwriter".

  ql/src/java/org/apache/hadoop/hive/ql/exec/ This method is called from
FileSinkOperator::jobCloseOp() and from RCFileMergeMapper::jobClose() both of which can get
access to reporter. I think we should make changes to propagate Reporter all the way through,
instead of passing null.
  ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/ Here also it
seems that reporter can be propagated in call hierachy from different join operators. So,
we should make those changes instead of passing null.



To: JIRA, ashutoshc, omalley

> getting the reporter in the recordwriter
> ----------------------------------------
>                 Key: HIVE-3234
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Serializers/Deserializers
>    Affects Versions: 0.9.1
>         Environment: any
>            Reporter: Jimmy Hu
>            Assignee: Owen O'Malley
>              Labels: newbie
>             Fix For: 0.9.1
>         Attachments: HIVE-3234.D6699.1.patch
>   Original Estimate: 48h
>  Remaining Estimate: 48h
> We would like to generate some custom statistics and report back to map/reduce later
wen implement the 
>  FileSinkOperator.RecordWriter interface. However, the current interface design doesn't
allow us to get the map reduce reporter object. Please extend the current FileSinkOperator.RecordWriter
interface so that it's close() method passes in a map reduce reporter object. 
> For the same reason, please also extend the RecordReader interface too to include a reporter
object so that users can passes in custom map reduce  counters.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message