hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-5634) use ReduceSinkOperator when outputting for client-side reducer
Date Wed, 23 Oct 2013 23:26:41 GMT

     [ https://issues.apache.org/jira/browse/HIVE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin updated HIVE-5634:
-----------------------------------

    Description: 
See HIVE-4002.
We can replace OutputCollector in ReduceSink to output to say a sequence file. Then instead
of fetching file output written by file sinks in map tasks, the client-side reducer can fetch
reducer output via some simple operator and work the same way as normal reducer.
It can also take advantage of additional ReduceSink functionality.

  was:
See HIVE-4002.
We can replace OutputCollector in ReduceSink to output to say a sequence file. Then instead
of fetching file output written by file sinks in map tasks, the client-side reducer can fetch
reducer output via some simple task and work the same way as normal reducer.
It can also take advantage of additional ReduceSink functionality.


> use ReduceSinkOperator when outputting for client-side reducer
> --------------------------------------------------------------
>
>                 Key: HIVE-5634
>                 URL: https://issues.apache.org/jira/browse/HIVE-5634
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>
> See HIVE-4002.
> We can replace OutputCollector in ReduceSink to output to say a sequence file. Then instead
of fetching file output written by file sinks in map tasks, the client-side reducer can fetch
reducer output via some simple operator and work the same way as normal reducer.
> It can also take advantage of additional ReduceSink functionality.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message