hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reynold Xin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-1211) Online aggregation and continuous query support
Date Fri, 21 Oct 2016 07:45:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594386#comment-15594386
] 

Reynold Xin commented on MAPREDUCE-1211:
----------------------------------------

This seems useful.


> Online aggregation and continuous query support
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1211
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1211
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: task
>            Reporter: Tyson Condie
>            Priority: Minor
>
> The purpose of this post is to propose a modified MapReduce architecture that allows
data to be pipelined between operators. This extends the MapReduce programming model beyond
batch processing, and can reduce completion times and improve system utilization for batch
jobs as well. We have built a modified version of the Hadoop MapReduce framework that supports
online aggregation, which allows users to see "early returns" from a job as it is being computed.
Our Hadoop Online Prototype (HOP) also supports continuous queries, which enable MapReduce
programs to be written for applications such as event monitoring and stream processing. HOP
retains the fault tolerance properties of Hadoop, and can run unmodified user-defined MapReduce
programs.
> For more information on the HOP design, please see our technical report.
> http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-136.html
> Further details are discussed in the following blog posts.
> http://databeta.wordpress.com/2009/10/18/mapreduce-online/
> http://radar.oreilly.com/2009/10/pipelining-and-real-time-analytics-with-mapreduce-online.html
> http://dbmsmusings.blogspot.com/2009/10/analysis-of-mapreduce-online-paper.html
> The HOP code has been published at the following location.
> http://code.google.com/p/hop/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message