Return-Path: X-Original-To: apmail-incubator-ambari-commits-archive@minotaur.apache.org Delivered-To: apmail-incubator-ambari-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 456A010D79 for ; Fri, 2 Aug 2013 00:39:08 +0000 (UTC) Received: (qmail 17505 invoked by uid 500); 2 Aug 2013 00:39:08 -0000 Delivered-To: apmail-incubator-ambari-commits-archive@incubator.apache.org Received: (qmail 17428 invoked by uid 500); 2 Aug 2013 00:39:08 -0000 Mailing-List: contact ambari-commits-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: ambari-dev@incubator.apache.org Delivered-To: mailing list ambari-commits@incubator.apache.org Received: (qmail 17217 invoked by uid 99); 2 Aug 2013 00:39:07 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Aug 2013 00:39:07 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 8D79D8B6A57; Fri, 2 Aug 2013 00:39:07 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: smohanty@apache.org To: ambari-commits@incubator.apache.org Date: Fri, 02 Aug 2013 00:39:12 -0000 Message-Id: <4dd6dda852a9459287ddc1896d186c5d@git.apache.org> In-Reply-To: <21ff8e603b2e41669e2ca2a82a104477@git.apache.org> References: <21ff8e603b2e41669e2ca2a82a104477@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [6/9] AMBARI-2794. Update stack version for 1.2.5. (smohanty) http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/hdfs-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/hdfs-site.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/hdfs-site.xml new file mode 100644 index 0000000..6246def --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/hdfs-site.xml @@ -0,0 +1,446 @@ + + + + + + + + + + + + + dfs.name.dir + + + Determines where on the local filesystem the DFS name node + should store the name table. If this is a comma-delimited list + of directories then the name table is replicated in all of the + directories, for redundancy. + true + + + + dfs.support.append + + to enable dfs append + true + + + + dfs.webhdfs.enabled + + to enable webhdfs + true + + + + dfs.datanode.socket.write.timeout + 0 + DFS Client write socket timeout + + + + dfs.datanode.failed.volumes.tolerated + + #of failed disks dn would tolerate + true + + + + dfs.block.local-path-access.user + + the user who is allowed to perform short + circuit reads. + + true + + + + dfs.data.dir + + Determines where on the local filesystem an DFS data node + should store its blocks. If this is a comma-delimited + list of directories, then data will be stored in all named + directories, typically on different devices. + Directories that do not exist are ignored. + + true + + + + dfs.hosts.exclude + + Names a file that contains a list of hosts that are + not permitted to connect to the namenode. The full pathname of the + file must be specified. If the value is empty, no hosts are + excluded. + + + + dfs.hosts + + Names a file that contains a list of hosts that are + permitted to connect to the namenode. The full pathname of the file + must be specified. If the value is empty, all hosts are + permitted. + + + + dfs.replication.max + 50 + Maximal block replication. + + + + + dfs.replication + + Default block replication. + + + + + dfs.heartbeat.interval + 3 + Determines datanode heartbeat interval in seconds. + + + + dfs.safemode.threshold.pct + 1.0f + + Specifies the percentage of blocks that should satisfy + the minimal replication requirement defined by dfs.replication.min. + Values less than or equal to 0 mean not to start in safe mode. + Values greater than 1 will make safe mode permanent. + + + + + dfs.balance.bandwidthPerSec + 6250000 + + Specifies the maximum amount of bandwidth that each datanode + can utilize for the balancing purpose in term of + the number of bytes per second. + + + + + dfs.datanode.address + + + + + dfs.datanode.http.address + + + + + dfs.block.size + 134217728 + The default block size for new files. + + + + dfs.http.address + +The name of the default file system. Either the +literal string "local" or a host:port for NDFS. +true + + + +dfs.datanode.du.reserved + + +Reserved space in bytes per volume. Always leave this much space free for non dfs use. + + + + +dfs.datanode.ipc.address +0.0.0.0:8010 + +The datanode ipc server address and port. +If the port is 0 then the server will start on a free port. + + + + +dfs.blockreport.initialDelay +120 +Delay for first block report in seconds. + + + +dfs.datanode.du.pct +0.85f +When calculating remaining space, only use this percentage of the real available space + + + + +dfs.namenode.handler.count +40 +The number of server threads for the namenode. + + + +dfs.datanode.max.xcievers +4096 +PRIVATE CONFIG VARIABLE + + + + + +dfs.umaskmode +077 + +The octal umask used when creating files and directories. + + + + +dfs.web.ugi + +gopher,gopher +The user account used by the web interface. +Syntax: USERNAME,GROUP1,GROUP2, ... + + + + +dfs.permissions +true + +If "true", enable permission checking in HDFS. +If "false", permission checking is turned off, +but all other behavior is unchanged. +Switching from one parameter value to the other does not change the mode, +owner or group of files or directories. + + + + +dfs.permissions.supergroup +hdfs +The name of the group of super-users. + + + +dfs.namenode.handler.count +100 +Added to grow Queue size so that more client connections are allowed + + + +ipc.server.max.response.size +5242880 + + +dfs.block.access.token.enable +true + +If "true", access tokens are used as capabilities for accessing datanodes. +If "false", no access tokens are checked on accessing datanodes. + + + + +dfs.namenode.kerberos.principal + + +Kerberos principal name for the NameNode + + + + +dfs.secondary.namenode.kerberos.principal + + + Kerberos principal name for the secondary NameNode. + + + + + + + dfs.namenode.kerberos.https.principal + + The Kerberos principal for the host that the NameNode runs on. + + + + + dfs.secondary.namenode.kerberos.https.principal + + The Kerberos principal for the hostthat the secondary NameNode runs on. + + + + + + dfs.secondary.http.address + + Address of secondary namenode web server + + + + dfs.secondary.https.port + 50490 + The https port where secondary-namenode binds + + + + dfs.web.authentication.kerberos.principal + + + The HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint. + The HTTP Kerberos principal MUST start with 'HTTP/' per Kerberos + HTTP SPENGO specification. + + + + + dfs.web.authentication.kerberos.keytab + + + The Kerberos keytab file with the credentials for the + HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint. + + + + + dfs.datanode.kerberos.principal + + + The Kerberos principal that the DataNode runs as. "_HOST" is replaced by the real host name. + + + + + dfs.namenode.keytab.file + + + Combined keytab file containing the namenode service and host principals. + + + + + dfs.secondary.namenode.keytab.file + + + Combined keytab file containing the namenode service and host principals. + + + + + dfs.datanode.keytab.file + + + The filename of the keytab file for the DataNode. + + + + + dfs.https.port + 50470 + The https port where namenode binds + + + + + dfs.https.address + + The https address where namenode binds + + + + + dfs.datanode.data.dir.perm + +The permissions that should be there on dfs.data.dir +directories. The datanode will not come up if the permissions are +different on existing dfs.data.dir directories. If the directories +don't exist, they will be created with this permission. + + + + dfs.access.time.precision + 0 + The access time for HDFS file is precise upto this value. + The default value is 1 hour. Setting a value of 0 disables + access times for HDFS. + + + + + dfs.cluster.administrators + hdfs + ACL for who all can view the default servlets in the HDFS + + + + ipc.server.read.threadpool.size + 5 + + + + + dfs.datanode.failed.volumes.tolerated + 0 + Number of failed disks datanode would tolerate + + + + dfs.namenode.avoid.read.stale.datanode + true + + Indicate whether or not to avoid reading from stale datanodes whose + heartbeat messages have not been received by the namenode for more than a + specified time interval. + + + + dfs.namenode.avoid.write.stale.datanode + true + + Indicate whether or not to avoid writing to stale datanodes whose + heartbeat messages have not been received by the namenode for more than a + specified time interval. + + + + dfs.namenode.write.stale.datanode.ratio + 1.0f + When the ratio of number stale datanodes to total datanodes marked is greater + than this ratio, stop avoiding writing to stale nodes so as to prevent causing hotspots. + + + + dfs.namenode.stale.datanode.interval + 30000 + Datanode is stale after not getting a heartbeat in this interval in ms + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/metainfo.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/metainfo.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/metainfo.xml new file mode 100644 index 0000000..8b61e0a --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/metainfo.xml @@ -0,0 +1,46 @@ + + + + root + Apache Hadoop Distributed File System + 1.2.0.1.3.1.0 + + + + NAMENODE + MASTER + + + + DATANODE + SLAVE + + + + SECONDARY_NAMENODE + MASTER + + + + HDFS_CLIENT + CLIENT + + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/global.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/global.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/global.xml new file mode 100644 index 0000000..d9adc80 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/global.xml @@ -0,0 +1,125 @@ + + + + + + + hivemetastore_host + + Hive Metastore host. + + + hive_database + + Hive database name. + + + hive_existing_mysql_database + + Hive database name. + + + hive_existing_mysql_host + + + + + hive_existing_oracle_database + + Hive database name. + + + hive_existing_oracle_host + + + + + hive_ambari_database + MySQL + Database type. + + + hive_ambari_host + + Database hostname. + + + hive_database_name + + Database hname + + + hive_metastore_user_name + hive + Database username to use to connect to the database. + + + hive_metastore_user_passwd + + Database password to use to connect to the database. + + + hive_metastore_port + 9083 + Hive Metastore port. + + + hive_lib + /usr/lib/hive/lib/ + Hive Library. + + + hive_dbroot + /usr/lib/hive/lib/ + Hive DB Directory. + + + hive_conf_dir + /etc/hive/conf + Hive Conf Dir. + + + hive_log_dir + /var/log/hive + Directory for Hive Log files. + + + hive_pid_dir + /var/run/hive + Hive PID Dir. + + + mysql_connector_url + ${download_url}/mysql-connector-java-5.1.18.zip + Hive PID Dir. + + + hive_aux_jars_path + /usr/lib/hcatalog/share/hcatalog/hcatalog-core.jar + Hive auxiliary jar path. + + + hive_user + hive + Hive User. + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-site.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-site.xml new file mode 100644 index 0000000..40fa0a7 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-site.xml @@ -0,0 +1,243 @@ + + + + + + + hive.metastore.local + false + controls whether to connect to remove metastore server or + open a new metastore server in Hive Client JVM + + + + javax.jdo.option.ConnectionURL + + JDBC connect string for a JDBC metastore + + + + javax.jdo.option.ConnectionDriverName + com.mysql.jdbc.Driver + Driver class name for a JDBC metastore + + + + javax.jdo.option.ConnectionUserName + + username to use against metastore database + + + + javax.jdo.option.ConnectionPassword + + password to use against metastore database + + + + hive.metastore.warehouse.dir + /apps/hive/warehouse + location of default database for the warehouse + + + + hive.metastore.sasl.enabled + + If true, the metastore thrift interface will be secured with SASL. + Clients must authenticate with Kerberos. + + + + hive.metastore.kerberos.keytab.file + + The path to the Kerberos Keytab file containing the metastore + thrift server's service principal. + + + + hive.metastore.kerberos.principal + + The service principal for the metastore thrift server. The special + string _HOST will be replaced automatically with the correct host name. + + + + hive.metastore.cache.pinobjtypes + Table,Database,Type,FieldSchema,Order + List of comma separated metastore object types that should be pinned in the cache + + + + hive.metastore.uris + + URI for client to contact metastore server + + + + hive.semantic.analyzer.factory.impl + org.apache.hivealog.cli.HCatSemanticAnalyzerFactory + controls which SemanticAnalyzerFactory implemenation class is used by CLI + + + + hadoop.clientside.fs.operations + true + FS operations are owned by client + + + + hive.metastore.client.socket.timeout + 60 + MetaStore Client socket timeout in seconds + + + + hive.metastore.execute.setugi + true + In unsecure mode, setting this property to true will cause the metastore to execute DFS operations using the client's reported user and group permissions. Note that this property must be set on both the client and server sides. Further note that its best effort. If client sets its to true and server sets it to false, client setting will be ignored. + + + + hive.security.authorization.enabled + true + enable or disable the hive client authorization + + + + hive.security.authorization.manager + org.apache.hcatalog.security.HdfsAuthorizationProvider + the hive client authorization manager class name. + The user defined authorization class should implement interface org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider. + + + + hive.server2.enable.doAs + true + + + + fs.hdfs.impl.disable.cache + true + + + + fs.file.impl.disable.cache + true + + + + hive.enforce.bucketing + true + Whether bucketing is enforced. If true, while inserting into the table, bucketing is enforced. + + + + hive.enforce.sorting + true + Whether sorting is enforced. If true, while inserting into the table, sorting is enforced. + + + + hive.map.aggr + true + Whether to use map-side aggregation in Hive Group By queries. + + + + hive.optimize.bucketmapjoin + true + + + + hive.optimize.bucketmapjoin.sortedmerge + true + + + + hive.mapred.reduce.tasks.speculative.execution + false + Whether speculative execution for reducers should be turned on. + + + + hive.auto.convert.join + true + Whether Hive enable the optimization about converting common + join into mapjoin based on the input file size. + + + + hive.auto.convert.sortmerge.join + true + Will the join be automatically converted to a sort-merge join, if the joined tables pass + the criteria for sort-merge join. + + + + + hive.auto.convert.sortmerge.join.noconditionaltask + true + + + + hive.auto.convert.join.noconditionaltask + true + Whether Hive enable the optimization about converting common join into mapjoin based on the input file + size. If this paramater is on, and the sum of size for n-1 of the tables/partitions for a n-way join is smaller than the + specified size, the join is directly converted to a mapjoin (there is no conditional task). + + + + + hive.auto.convert.join.noconditionaltask.size + 1000000000 + If hive.auto.convert.join.noconditionaltask is off, this parameter does not take affect. However, if it + is on, and the sum of size for n-1 of the tables/partitions for a n-way join is smaller than this size, the join is directly + converted to a mapjoin(there is no conditional task). The default is 10MB. + + + + + hive.optimize.reducededuplication.min.reducer + 1 + Reduce deduplication merges two RSs by moving key/parts/reducer-num of the child RS to parent RS. + That means if reducer-num of the child RS is fixed (order by or forced bucketing) and small, it can make very slow, single MR. + The optimization will be disabled if number of reducers is less than specified value. + + + + + hive.optimize.mapjoin.mapreduce + true + If hive.auto.convert.join is off, this parameter does not take + affect. If it is on, and if there are map-join jobs followed by a map-reduce + job (for e.g a group by), each map-only job is merged with the following + map-reduce job. + + + + + hive.mapjoin.bucket.cache.size + 10000 + + Size per reducer.The default is 1G, i.e if the input size is 10G, it + will use 10 reducers. + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/metainfo.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/metainfo.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/metainfo.xml new file mode 100644 index 0000000..b185e13 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/metainfo.xml @@ -0,0 +1,43 @@ + + + + root + Data warehouse system for ad-hoc queries & analysis of large datasets and table & storage management service + 0.11.0.1.3.1.0 + + + + HIVE_METASTORE + MASTER + + + HIVE_SERVER + MASTER + + + MYSQL_SERVER + MASTER + + + HIVE_CLIENT + CLIENT + + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/global.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/global.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/global.xml new file mode 100644 index 0000000..c49480f --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/global.xml @@ -0,0 +1,35 @@ + + + + + + + hue_pid_dir + /var/run/hue + Hue Pid Dir. + + + hue_log_dir + /var/log/hue + Hue Log Dir. + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/hue-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/hue-site.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/hue-site.xml new file mode 100644 index 0000000..6eb52a2 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/configuration/hue-site.xml @@ -0,0 +1,290 @@ + + + + + + + + send_debug_messages + 1 + + + + + database_logging + 0 + To show database transactions, set database_logging to 1. + default, database_logging=0 + + + + secret_key + + This is used for secure hashing in the session store. + + + + http_host + 0.0.0.0 + Webserver listens on this address and port + + + + http_port + 8000 + Webserver listens on this address and port + + + + time_zone + America/Los_Angeles + Time zone name + + + + django_debug_mode + 1 + Turn off debug + + + + use_cherrypy_server + false + Set to true to use CherryPy as the webserver, set to false + to use Spawning as the webserver. Defaults to Spawning if + key is not specified. + + + + http_500_debug_mode + 1 + Turn off backtrace for server error + + + + server_user + + Webserver runs as this user + + + + server_group + + Webserver runs as this user + + + + backend_auth_policy + desktop.auth.backend.AllowAllBackend + Authentication backend. + + + + + db_engine + mysql + Configuration options for specifying the Desktop Database. + + + + db_host + localhost + Configuration options for specifying the Desktop Database. + + + + db_port + 3306 + Configuration options for specifying the Desktop Database. + + + + db_user + sandbox + Configuration options for specifying the Desktop Database. + + + + db_password + 1111 + Configuration options for specifying the Desktop Database. + + + + db_name + sandbox + Configuration options for specifying the Desktop Database. + + + + + smtp_host + localhost + The SMTP server information for email notification delivery. + + + + smtp_port + 25 + The SMTP server information for email notification delivery. + + + + smtp_user + + The SMTP server information for email notification delivery. + + + + smtp_password + 25 + The SMTP server information for email notification delivery. + + + + tls + no + Whether to use a TLS (secure) connection when talking to the SMTP server. + + + + default_from_email + sandbox@hortonworks.com + The SMTP server information for email notification delivery. + + + + + fs_defaultfs + + Enter the filesystem uri. E.g + .:hdfs://sandbox:8020 + + + + webhdfs_url + + Use WebHdfs/HttpFs as the communication mechanism. To fallback to + using the Thrift plugin (used in Hue 1.x), this must be uncommented + and explicitly set to the empty value. + Value e.g.: http://localhost:50070/webhdfs/v1/ + + + + jobtracker_host + + Enter the host on which you are running the Hadoop JobTracker. + + + + jobtracker_port + 50030 + The port where the JobTracker IPC listens on. + + + + hadoop_mapred_home + /usr/lib/hadoop/lib + The SMTP server information for email notification delivery. + + + + resourcemanager_host + + Enter the host on which you are running the ResourceManager. + + + + resourcemanager_port + + The port where the ResourceManager IPC listens on. + + + + + hive_home_dir + + Hive home directory. + + + + hive_conf_dir + + Hive configuration directory, where hive-site.xml is + located. + + + + templeton_url + + WebHcat http URL + + + + + pig_nice_name + + Define and configure a new shell type pig + + + + pig_shell_command + /usr/bin/pig -l /dev/null + Define and configure a new shell type pig. + + + + pig_java_home + + Define and configure a new shell type pig. + + + + hbase_nice_name + HBase Shell + Define and configure a new shell type hbase + + + + hbase_shell_command + /usr/bin/hbase shell + Define and configure a new shell type hbase. + + + + bash_nice_name + + Define and configure a new shell type bash for testing + only + + + + bash_shell_command + /bin/bash + Define and configure a new shell type bash for testing only + . + + + + + whitelist + (localhost|127\.0\.0\.1):(50030|50070|50060|50075|50111) + proxy settings + + + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/metainfo.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/metainfo.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/metainfo.xml new file mode 100644 index 0000000..c6e384f --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HUE/metainfo.xml @@ -0,0 +1,31 @@ + + + + root + Hue is a graphical user interface to operate and develop + applications for Apache Hadoop. + 2.2.0 + + + + HUE_SERVER + MASTER + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/capacity-scheduler.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/capacity-scheduler.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/capacity-scheduler.xml new file mode 100644 index 0000000..8034d19 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/capacity-scheduler.xml @@ -0,0 +1,195 @@ + + + + + + + + + + + + + mapred.capacity-scheduler.maximum-system-jobs + 3000 + Maximum number of jobs in the system which can be initialized, + concurrently, by the CapacityScheduler. + + + + + mapred.capacity-scheduler.queue.default.capacity + 100 + Percentage of the number of slots in the cluster that are + to be available for jobs in this queue. + + + + + mapred.capacity-scheduler.queue.default.maximum-capacity + -1 + + maximum-capacity defines a limit beyond which a queue cannot use the capacity of the cluster. + This provides a means to limit how much excess capacity a queue can use. By default, there is no limit. + The maximum-capacity of a queue can only be greater than or equal to its minimum capacity. + Default value of -1 implies a queue can use complete capacity of the cluster. + + This property could be to curtail certain jobs which are long running in nature from occupying more than a + certain percentage of the cluster, which in the absence of pre-emption, could lead to capacity guarantees of + other queues being affected. + + One important thing to note is that maximum-capacity is a percentage , so based on the cluster's capacity + the max capacity would change. So if large no of nodes or racks get added to the cluster , max Capacity in + absolute terms would increase accordingly. + + + + + mapred.capacity-scheduler.queue.default.supports-priority + false + If true, priorities of jobs will be taken into + account in scheduling decisions. + + + + + mapred.capacity-scheduler.queue.default.minimum-user-limit-percent + 100 + Each queue enforces a limit on the percentage of resources + allocated to a user at any given time, if there is competition for them. + This user limit can vary between a minimum and maximum value. The former + depends on the number of users who have submitted jobs, and the latter is + set to this property value. For example, suppose the value of this + property is 25. If two users have submitted jobs to a queue, no single + user can use more than 50% of the queue resources. If a third user submits + a job, no single user can use more than 33% of the queue resources. With 4 + or more users, no user can use more than 25% of the queue's resources. A + value of 100 implies no user limits are imposed. + + + + + mapred.capacity-scheduler.queue.default.user-limit-factor + 1 + The multiple of the queue capacity which can be configured to + allow a single user to acquire more slots. + + + + + mapred.capacity-scheduler.queue.default.maximum-initialized-active-tasks + 200000 + The maximum number of tasks, across all jobs in the queue, + which can be initialized concurrently. Once the queue's jobs exceed this + limit they will be queued on disk. + + + + + mapred.capacity-scheduler.queue.default.maximum-initialized-active-tasks-per-user + 100000 + The maximum number of tasks per-user, across all the of the + user's jobs in the queue, which can be initialized concurrently. Once the + user's jobs exceed this limit they will be queued on disk. + + + + + mapred.capacity-scheduler.queue.default.init-accept-jobs-factor + 10 + The multipe of (maximum-system-jobs * queue-capacity) used to + determine the number of jobs which are accepted by the scheduler. + + + + + + + + mapred.capacity-scheduler.default-supports-priority + false + If true, priorities of jobs will be taken into + account in scheduling decisions by default in a job queue. + + + + + mapred.capacity-scheduler.default-minimum-user-limit-percent + 100 + The percentage of the resources limited to a particular user + for the job queue at any given point of time by default. + + + + + + mapred.capacity-scheduler.default-user-limit-factor + 1 + The default multiple of queue-capacity which is used to + determine the amount of slots a single user can consume concurrently. + + + + + mapred.capacity-scheduler.default-maximum-active-tasks-per-queue + 200000 + The default maximum number of tasks, across all jobs in the + queue, which can be initialized concurrently. Once the queue's jobs exceed + this limit they will be queued on disk. + + + + + mapred.capacity-scheduler.default-maximum-active-tasks-per-user + 100000 + The default maximum number of tasks per-user, across all the of + the user's jobs in the queue, which can be initialized concurrently. Once + the user's jobs exceed this limit they will be queued on disk. + + + + + mapred.capacity-scheduler.default-init-accept-jobs-factor + 10 + The default multipe of (maximum-system-jobs * queue-capacity) + used to determine the number of jobs which are accepted by the scheduler. + + + + + + mapred.capacity-scheduler.init-poll-interval + 5000 + The amount of time in miliseconds which is used to poll + the job queues for jobs to initialize. + + + + mapred.capacity-scheduler.init-worker-threads + 5 + Number of worker threads which would be used by + Initialization poller to initialize jobs in a set of queue. + If number mentioned in property is equal to number of job queues + then a single thread would initialize jobs in a queue. If lesser + then a thread would get a set of queues assigned. If the number + is greater then number of threads would be equal to number of + job queues. + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/core-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/core-site.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/core-site.xml new file mode 100644 index 0000000..3a2af49 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/core-site.xml @@ -0,0 +1,20 @@ + + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/global.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/global.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/global.xml new file mode 100644 index 0000000..d279724 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/global.xml @@ -0,0 +1,160 @@ + + + + + + + jobtracker_host + + JobTracker Host. + + + tasktracker_hosts + + TaskTracker hosts. + + + mapred_local_dir + /hadoop/mapred + MapRed Local Directories. + + + mapred_system_dir + /mapred/system + MapRed System Directories. + + + scheduler_name + org.apache.hadoop.mapred.CapacityTaskScheduler + MapRed Capacity Scheduler. + + + jtnode_opt_newsize + 200 + Mem New Size. + + + jtnode_opt_maxnewsize + 200 + Max New size. + + + hadoop_heapsize + 1024 + Hadoop maximum Java heap size + + + jtnode_heapsize + 1024 + Maximum Java heap size for JobTracker in MB (Java option -Xmx) + + + mapred_map_tasks_max + 4 + Number of slots that Map tasks that run simultaneously can occupy on a TaskTracker + + + mapred_red_tasks_max + 2 + Number of slots that Reduce tasks that run simultaneously can occupy on a TaskTracker + + + mapred_cluster_map_mem_mb + -1 + The virtual memory size of a single Map slot in the MapReduce framework + + + mapred_cluster_red_mem_mb + -1 + The virtual memory size of a single Reduce slot in the MapReduce framework + + + mapred_job_map_mem_mb + -1 + Virtual memory for single Map task + + + mapred_child_java_opts_sz + 768 + Java options for the TaskTracker child processes. + + + io_sort_mb + 200 + The total amount of Map-side buffer memory to use while sorting files (Expert-only configuration). + + + io_sort_spill_percent + 0.9 + Percentage of sort buffer used for record collection (Expert-only configuration. + + + mapreduce_userlog_retainhours + 24 + The maximum time, in hours, for which the user-logs are to be retained after the job completion. + + + maxtasks_per_job + -1 + Maximum number of tasks for a single Job + + + lzo_enabled + false + LZO compression enabled + + + snappy_enabled + true + LZO compression enabled + + + rca_enabled + true + Enable Job Diagnostics. + + + mapred_hosts_exclude + + Exclude entered hosts + + + mapred_hosts_include + + Include entered hosts + + + mapred_jobstatus_dir + file:////mapred/jobstatus + Job Status directory + + + task_controller + org.apache.hadoop.mapred.DefaultTaskController + Task Controller. + + + mapred_user + mapred + MapReduce User. + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-queue-acls.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-queue-acls.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-queue-acls.xml new file mode 100644 index 0000000..ce12380 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-queue-acls.xml @@ -0,0 +1,39 @@ + + + + + + + + + + + + + mapred.queue.default.acl-submit-job + * + + + + mapred.queue.default.acl-administer-jobs + * + + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-site.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-site.xml new file mode 100644 index 0000000..df6ca71 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/mapred-site.xml @@ -0,0 +1,537 @@ + + + + + + + + + + + + + io.sort.mb + + No description + + + + io.sort.record.percent + .2 + No description + + + + io.sort.spill.percent + + No description + + + + io.sort.factor + 100 + No description + + + + + + mapred.tasktracker.tasks.sleeptime-before-sigkill + 250 + Normally, this is the amount of time before killing + processes, and the recommended-default is 5.000 seconds - a value of + 5000 here. In this case, we are using it solely to blast tasks before + killing them, and killing them very quickly (1/4 second) to guarantee + that we do not leave VMs around for later jobs. + + + + + mapred.job.tracker.handler.count + 50 + + The number of server threads for the JobTracker. This should be roughly + 4% of the number of tasktracker nodes. + + + + + mapred.system.dir + /mapred/system + No description + true + + + + mapred.job.tracker + + + No description + true + + + + mapred.job.tracker.http.address + + + No description + true + + + + + mapred.local.dir + + No description + true + + + + mapreduce.cluster.administrators + hadoop + + + + mapred.reduce.parallel.copies + 30 + No description + + + + mapred.tasktracker.map.tasks.maximum + + No description + + + + mapred.tasktracker.reduce.tasks.maximum + + No description + + + + tasktracker.http.threads + 50 + + + + mapred.map.tasks.speculative.execution + false + If true, then multiple instances of some map tasks + may be executed in parallel. + + + + mapred.reduce.tasks.speculative.execution + false + If true, then multiple instances of some reduce tasks + may be executed in parallel. + + + + mapred.reduce.slowstart.completed.maps + 0.05 + + + + mapred.inmem.merge.threshold + 1000 + The threshold, in terms of the number of files + for the in-memory merge process. When we accumulate threshold number of files + we initiate the in-memory merge and spill to disk. A value of 0 or less than + 0 indicates we want to DON'T have any threshold and instead depend only on + the ramfs's memory consumption to trigger the merge. + + + + + mapred.job.shuffle.merge.percent + 0.66 + The usage threshold at which an in-memory merge will be + initiated, expressed as a percentage of the total memory allocated to + storing in-memory map outputs, as defined by + mapred.job.shuffle.input.buffer.percent. + + + + + mapred.job.shuffle.input.buffer.percent + 0.7 + The percentage of memory to be allocated from the maximum heap + size to storing map outputs during the shuffle. + + + + + mapred.map.output.compression.codec + org.apache.hadoop.io.compress.SnappyCodec + If the map outputs are compressed, how should they be + compressed + + + + + mapred.output.compression.type + BLOCK + If the job outputs are to compressed as SequenceFiles, how should + they be compressed? Should be one of NONE, RECORD or BLOCK. + + + + + + mapred.jobtracker.completeuserjobs.maximum + 5 + + + + mapred.jobtracker.taskScheduler + + + + + mapred.jobtracker.restart.recover + false + "true" to enable (job) recovery upon restart, + "false" to start afresh + + + + + mapred.job.reduce.input.buffer.percent + 0.0 + The percentage of memory- relative to the maximum heap size- to + retain map outputs during the reduce. When the shuffle is concluded, any + remaining map outputs in memory must consume less than this threshold before + the reduce can begin. + + + + + mapreduce.reduce.input.limit + 10737418240 + The limit on the input size of the reduce. (This value + is 10 Gb.) If the estimated input size of the reduce is greater than + this value, job is failed. A value of -1 means that there is no limit + set. + + + + + + mapred.compress.map.output + + + + + + mapred.task.timeout + 600000 + The number of milliseconds before a task will be + terminated if it neither reads an input, writes an output, nor + updates its status string. + + + + + jetty.connector + org.mortbay.jetty.nio.SelectChannelConnector + No description + + + + mapred.task.tracker.task-controller + org.apache.hadoop.mapred.DefaultTaskController + + TaskController which is used to launch and manage task execution. + + + + + mapred.child.root.logger + INFO,TLA + + + + mapred.child.java.opts + + + No description + + + + mapred.cluster.map.memory.mb + 1536 + + + + mapred.cluster.reduce.memory.mb + 2048 + + + + mapred.job.map.memory.mb + 1536 + + + + mapred.job.reduce.memory.mb + 2048 + + + + mapred.cluster.max.map.memory.mb + 6144 + + + + mapred.cluster.max.reduce.memory.mb + 4096 + + + + mapred.hosts + + + + + mapred.hosts.exclude + + + + + mapred.max.tracker.blacklists + 16 + + if node is reported blacklisted by 16 successful jobs within timeout-window, it will be graylisted + + + + + mapred.healthChecker.script.path + + + + + mapred.healthChecker.interval + 135000 + + + + mapred.healthChecker.script.timeout + 60000 + + + + mapred.job.tracker.persist.jobstatus.active + false + Indicates if persistency of job status information is + active or not. + + + + + mapred.job.tracker.persist.jobstatus.hours + 1 + The number of hours job status information is persisted in DFS. + The job status information will be available after it drops of the memory + queue and between jobtracker restarts. With a zero value the job status + information is not persisted at all in DFS. + + + + + mapred.job.tracker.persist.jobstatus.dir + + The directory where the job status information is persisted + in a file system to be available after it drops of the memory queue and + between jobtracker restarts. + + + + + mapred.jobtracker.retirejob.check + 10000 + + + + mapred.jobtracker.retirejob.interval + 21600000 + + + + mapred.job.tracker.history.completed.location + /mapred/history/done + No description + + + + mapred.task.maxvmem + + true + No description + + + + mapred.jobtracker.maxtasks.per.job + + true + The maximum number of tasks for a single job. + A value of -1 indicates that there is no maximum. + + + + mapreduce.fileoutputcommitter.marksuccessfuljobs + false + + + + mapred.userlog.retain.hours + + + + + mapred.job.reuse.jvm.num.tasks + 1 + + How many tasks to run per jvm. If set to -1, there is no limit + + true + + + + mapreduce.jobtracker.kerberos.principal + + + JT user name key. + + + + + mapreduce.tasktracker.kerberos.principal + + + tt user name key. "_HOST" is replaced by the host name of the task tracker. + + + + + + hadoop.job.history.user.location + none + true + + + + + mapreduce.jobtracker.keytab.file + + + The keytab for the jobtracker principal. + + + + + + mapreduce.tasktracker.keytab.file + + The filename of the keytab for the task tracker + + + + mapred.task.tracker.http.address + + Http address for task tracker. + + + + mapreduce.jobtracker.staging.root.dir + /user + The Path prefix for where the staging directories should be placed. The next level is always the user's + name. It is a path in the default file system. + + + + mapreduce.tasktracker.group + hadoop + The group that the task controller uses for accessing the task controller. The mapred user must be a member and users should *not* be members. + + + + + mapreduce.jobtracker.split.metainfo.maxsize + 50000000 + true + If the size of the split metainfo file is larger than this, the JobTracker will fail the job during + initialize. + + + + mapreduce.history.server.embedded + false + Should job history server be embedded within Job tracker +process + true + + + + mapreduce.history.server.http.address + + + Http address of the history server + true + + + + mapreduce.jobhistory.kerberos.principal + + + Job history user name key. (must map to same user as JT +user) + + + + mapreduce.jobhistory.keytab.file + + + The keytab for the job history server principal. + + + + mapred.jobtracker.blacklist.fault-timeout-window + 180 + + 3-hour sliding window (value is in minutes) + + + + + mapred.jobtracker.blacklist.fault-bucket-width + 15 + + 15-minute bucket size (value is in minutes) + + + + + mapred.queue.names + default + Comma separated list of queues configured for this jobtracker. + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/metainfo.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/metainfo.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/metainfo.xml new file mode 100644 index 0000000..6107e54 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/metainfo.xml @@ -0,0 +1,41 @@ + + + + mapred + Apache Hadoop Distributed Processing Framework + 1.2.0.1.3.1.0 + + + + JOBTRACKER + MASTER + + + + TASKTRACKER + SLAVE + + + + MAPREDUCE_CLIENT + CLIENT + + + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/configuration/global.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/configuration/global.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/configuration/global.xml new file mode 100644 index 0000000..61a2b90 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/configuration/global.xml @@ -0,0 +1,50 @@ + + + + + + + nagios_user + nagios + Nagios Username. + + + nagios_group + nagios + Nagios Group. + + + nagios_web_login + nagiosadmin + Nagios web user. + + + nagios_web_password + + Nagios Admin Password. + + + nagios_contact + + Hadoop Admin Email. + + + http://git-wip-us.apache.org/repos/asf/incubator-ambari/blob/e67c8112/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/metainfo.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/metainfo.xml b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/metainfo.xml new file mode 100644 index 0000000..bd7de07 --- /dev/null +++ b/ambari-server/src/main/resources/stacks/HDP/1.3.2/services/NAGIOS/metainfo.xml @@ -0,0 +1,30 @@ + + + + root + Nagios Monitoring and Alerting system + 3.2.3 + + + + NAGIOS_SERVER + MASTER + + + +