hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wiley <kwi...@keithwiley.com>
Subject Re: Oozie vs. YARN "application"
Date Fri, 18 May 2012 00:41:53 GMT
Just to be clear yarn does offer something described as dag management, but perhaps you mean
it doesn't offer a dag *of* applications, only of mr jobs *within* an application. Is that
what you mean?
Sent from my phone, please excuse my brevity.
Keith Wiley, kwiley@keithwiley.com, http://keithwiley.com

Arun C Murthy <acm@hortonworks.com> wrote:

YARN doesn't yet have a DAGApplicationMaster which can handle a DAG of jobs.

Conceivable Oozie could offload the DAG management once it's available.

OTOH, Oozie provides much more than just DAG management - it provides time & data-availability
based scheduling of workflows.


On May 17, 2012, at 4:47 PM, Keith Wiley wrote:

> Hadoop 0.23 (MRv2 or YARN) provides the concept of an "application", which is described
as either a classic single MR job *or* a DAG of such jobs...which at a glance appears redundant
with Oozie's primary purpose.
> How do they differ? Does 0.23 effectively obviate the service that Oozie provides or
is Oozie more powerful than a YARN "application"...or simply more different from an application
than I have conceptualized to the point that they don't necessarily step on one another's

> Keith Wiley kwiley@keithwiley.com keithwiley.com music.keithwiley.com
> "The easy confidence with which I know another man's religion is folly teaches
> me to suspect that my own is also."
> -- Mark Twain


Arun C. Murthy
Hortonworks Inc.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message