incubator-ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: stack definition and component plugin
Date Tue, 17 Jan 2012 07:40:13 GMT
On Mon, Jan 16, 2012 at 10:25 PM, Devaraj Das <ddas@hortonworks.com> wrote:

> Don't understand this. Every component knows what components it depends on, right? MapRed
component developer would know that it depends on hdfs. If you depend on hbase, you should
ensure that the list of dependent components includes hbase...

Mapreduce can depend on either HDFS only, or HDFS+HBase+ZooKeeper.
(For people who runs mapreduce on top of HBase).  Therefore, the
developer for implementing mapreduce plugin needs to handle both cases
in the plugin.  It is also possible to run mapreduce on other systems
like cassandra, or mongodb.  Hence, the permutation for conditions in
mapreduce plugin becomes too complex, and there could be different
ways to handle mapreduce 1.0 or YARN.  The current plugin design does
not provide good visibility on how to handle the conditions.

> Actually, Ambari could get out of the business of orchestration. Instead, the components
should be updated to tolerate their dependencies being down.. I have heard this from some
people, and generally concur with this thinking. It makes the ecosystem better (if you ignore
Ambari for the moment).

It seems unsafe to make this assumption.  The eventual consistent
model might not be a good idea.  Remember the example that I suggested
a while back, a old datanode might be doing hand shake with a newer
version of namenode, and accepted data for writing.  This might be
causing data instability because the system is in transition of
upgrade.  There could be unexpected bugs in software that breaks
eventual consistent model.  I would recommend to think twice with
going this approach.

regards,
Eric

Mime
View raw message