ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Swan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-1783) Specification for a cluster blueprint consumable by Ambari
Date Tue, 16 Apr 2013 20:45:16 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633330#comment-13633330
] 

Brian Swan commented on AMBARI-1783:
------------------------------------

This is a great start. Some thoughts...

• Allowing the various sections (Stack Def., Stack Conf., Host Manifest, Host-Component
Mapping) to be consumable separately would be helpful for a number of reasons:
		○ Stack Def will be written by a stack author. A cluster admin (someone deploying a cluster)
does not need to be concerned with these. He *does* need to be concerned with the Host Manifest
and Host-Component Mapping (which describe the topology of the cluster).
		○ In some cases, host details (as specified in the Host Manifest) will be controlled by
other tools (such as System Center Virtual Machine Manger or an agent in a Cloud environment).
		○ In some cases, Host-Component Mapping will be controlled by a tool (such as System Center
Virtual Machine Manager, or an agent in a Cloud environment) and will not depend on this content
in the blueprint (i.e. it needs to be ignorable).
		○ Introducing a level of abstraction to the Host Manifest and Host-Component Mapping content
that allows a cluster admin to define the topology of a cluster without specifying host details
of mapping details would help with the two previous points. For example:

			"Node": [
			        {
			            "Name": "Master",
			            "Count": 1,
			            "Component": [ "openjdk7", "core", "namenode", "resourcemanager"],
			            "Type": "HeadNode"
			        },
			        {
			            "Name": "Slave",
			            "Count": 4,
			            "Component": [ "openjdk7", "core", "datanode", "nodemanager", "drawbridge"],
			            "Type": "WorkerNode"
			        }
			    ],
			

	• Providing for an option to instal custom software (such as Python or a specific version
of the JDK) across a cluster could be very useful. This could be as simple as a reference
to a script that knows how to install software. For example, this could be part of the Stack
Definition (though this doesn't adhere exactly to the proposed syntax):

		"Service": [
		        {
		            "Name": "OracleJDK",
		            "Component": [
		                {
		                    "Cardinality": "*",
		                    "IsRequired": false,
		                    "IsServerComponent": false,
		                    "Name": "OracleJDK",
		                    "Package": {
		                        "Type": "custom",
		                        "Name": "OracleJDK",
		                        "ScriptFileName": "OracleJDK.ps1"
		                    }
		                }
		            ]
		        },
		
	• Resource URIs in the Stack Configuration would allow for identification of where distributions
(whether they be Hadoop distributions or custom software) could be downloaded. In the example
below, the Resources identify where the hdinsight and python packages should be downloaded
from:

		"Distros": [
		        { "Name": "hdinsight", "Version": "1.0.1", "FileName": "hdp-azure-1.0.1.package.zip"
},
			{ "Name": "python", "Version": "2.6", "FileName": "python.zip"}
		    ],
		    "Resources": [
			{"HDP":"http://mycontainer.blob.core.windows.net/hdp-azure-1.01.package.zip"},
			{"Python":"http://mycontainer.blob.core.windows.net/python.zip"}]
		    ],
		
		
	• A Users section in the Stack Configuration would allow for users to be created on machines
in the cluster as they are provisioned.  Hadoop services would run as these users. This section
could be ignored when it isn't needed.
		"Users": [ 
		        { 
		            "Groups": [ 
		                "hadoopadmin" 
		            ], 
		            "Name": "hadoopuser1", 
		            "Password": "pwd", 
		            "Type": "local" 
		        } 
		    ],
		

                
> Specification for a cluster blueprint consumable by Ambari
> ----------------------------------------------------------
>
>                 Key: AMBARI-1783
>                 URL: https://issues.apache.org/jira/browse/AMBARI-1783
>             Project: Ambari
>          Issue Type: New Feature
>    Affects Versions: 1.3.1
>            Reporter: Sumit Mohanty
>            Assignee: Sumit Mohanty
>             Fix For: 1.3.1
>
>         Attachments: Ambari-BluePrint-0.1.docx, CompleteBlueprint.txt, HostComponentMapping.txt,
HostManifest.txt, StackConfiguration.txt, StackDefinition.txt
>
>
> Deployment of a hadoop cluster can be modeled as the mapping of a stack definition to
a set of host while satisfying placement constraints. A stack definition refers to the description
of various services that comprise a hadoop stack (e.g. HDP-1.2.1), components associated with
the services, and configuration properties. A set of available hosts is specified through
a host-manifest that lists the available hosts and relevant properties/characteristics of
each host. A cluster blueprint is specification of a hadoop stack mapped to a set of hosts.
Mapping to a host can be a direct mapping - e.g. deploy this component X on host Y or a host
can be specified as a set of requirements which is used to select the most appropriate host(s)
from a host-manifest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message