incubator-ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject stack definition and component plugin
Date Tue, 17 Jan 2012 06:08:43 GMT
Hi all,

The current hierarchical stack definition is confusing to me.
Supposedly, the definition can be expanded and flatten configuration
key value pair, and an example of defining namenode url and get
inherited in hbase configuration would look like this:

{
  ...
  "components": {
    "hdfs": {
      "roles": {
        "namenode": { /* override one value on the namenode */
          "hadoop/hdfs-site": {
            "dfs.https.enable": "true",
            "fs.default.name": "hdfs://${namenode}:${port}/"
          }
        }
      }
    },
    "hbase": {
      "roles": {
        "region-server": {
          "hbase/hbase-site": {
            "hbase.rootdir":
"${components.hdfs.namenode.hadoop/hdfs-site.fs.default.name}/hbase"
        }
      }
    }
  }
}

hbase.rootdir is a key for hbase-site.xml, and it should contain key
of "fs.default.name" plus additional path for hbase to store data.  In
my interpretation of the macro would look like
${components.hdfs.namenode.hadoop/hdfs-site.fs.default.name}/hbase.
This seems like a utterly awkward method of describing inheritance.
Why don't we use a flat name space to remove additional logistics
imposed by Ambari.  I agree that the syntax is fully accurate, but it
is a larger headache to maintain this hierarchical structure.

The second problem is the component plugin architecture sounds good in
theory.  I see some scalability issues with this approach.  Each
component describes the components that it depends on.  This could
interfere with introducing new components.  i.e. Mapreduce component
depends on HDFS.  A new component is introduced, and name HBase.  Now,
the Mapreduce component needs to update it's dependency to HDFS and
HBase and ZooKeeper.  For introducing new component, there is a lot of
plugins updates to make the new version work.  The plugin writer also
needs to make theoretical assumption that if components X is installed
do Y, otherwise do Z.  Conditional assumption in plugin introduces
uncertainty and corner cases into the deployment system.  The number
of permutation can greatly exceed the logics that is required to
handle by the plugin.

Instead of using plugin architecture to manage deployment, it would be
safer to use a scripting approach to enable power administrator to
deploy a stack of software by writing shell script like script to
accomplish the deployment tasks.  The recipes scripts can be shared by
the community to automate software stack deployment.  This will ensure
the scope of Ambari deployment is focus on cross nodes orchestration
without having to build bell and whistles which does not scale well in
the long term.

What do you guys think?

regards,
Eric

Mime
View raw message