hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1107) Generic parallel execution framework for Hive (and Pig, and ...)
Date Mon, 11 Apr 2011 23:34:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018630#comment-13018630
] 

Carl Steinbach commented on HIVE-1107:
--------------------------------------

bq. Hive/Pig might want to change the #reduces of job3 in the pipeline after looking at the
output of job1 and job2 - this is not necessarily changing the DAG itself, but changing the
components of the DAG.

Oozie already supports this:
http://yahoo.github.com/oozie/releases/2.0.0/WorkflowFunctionalSpec.html#DecisionNode
http://yahoo.github.com/oozie/releases/2.0.0/WorkflowFunctionalSpec.html#WorkflowELSupport


> Generic parallel execution framework for Hive (and Pig, and ...)
> ----------------------------------------------------------------
>
>                 Key: HIVE-1107
>                 URL: https://issues.apache.org/jira/browse/HIVE-1107
>             Project: Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Carl Steinbach
>
> Pig and Hive each have their own libraries for handling plan execution. As we prepare
to invest more time improving Hive's plan execution mechanism we should also start to consider
ways of building a generic plan execution mechanism that is capable of supporting the needs
of Hive and Pig, as well as other Hadoop data flow programming environments. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message