hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From inigo...@apache.org
Subject [2/2] hadoop git commit: YARN-8389. Improve the description of machine-list property in Federation docs. Contributed by Takanobu Asanuma.
Date Mon, 04 Jun 2018 16:29:08 GMT
YARN-8389. Improve the description of machine-list property in Federation docs. Contributed
by Takanobu Asanuma.

(cherry picked from commit 61fc7f73f21b0949e27ef3893efda757d91a03f9)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/83d0964d
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/83d0964d
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/83d0964d

Branch: refs/heads/branch-3.1
Commit: 83d0964d827b86df800f918e884a3897e33ac613
Parents: 6646b16
Author: Inigo Goiri <inigoiri@apache.org>
Authored: Mon Jun 4 09:28:09 2018 -0700
Committer: Inigo Goiri <inigoiri@apache.org>
Committed: Mon Jun 4 09:28:40 2018 -0700

----------------------------------------------------------------------
 .../hadoop-yarn-site/src/site/markdown/Federation.md          | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/83d0964d/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md
index a1d3ab6..087a5b0 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md
@@ -218,7 +218,7 @@ SQL-Server scripts are located in **sbin/FederationStateStore/SQLServer/**.
 |`yarn.federation.policy-manager` | `org.apache.hadoop.yarn.server.federation.policies.manager.WeightedLocalityPolicyManager`
| The choice of policy manager determines how Applications and ResourceRequests are routed
through the system. |
 |`yarn.federation.policy-manager-params` | `<binary>` | The payload that configures
the policy. In our example a set of weights for router and amrmproxy policies. This is typically
generated by serializing a policymanager that has been configured programmatically, or by
populating the state-store with the .json serialized form of it. |
 |`yarn.federation.subcluster-resolver.class` | `org.apache.hadoop.yarn.server.federation.resolver.DefaultSubClusterResolverImpl`
| The class used to resolve which subcluster a node belongs to, and which subcluster(s) a
rack belongs to. |
-| `yarn.federation.machine-list` | `node1,subcluster1,rack1\n node2 , subcluster2, RACK1\n
node3,subcluster3, rack2\n node4, subcluster3, rack2\n` | a list of Nodes, Sub-clusters, Rack,
used by the `DefaultSubClusterResolverImpl` |
+|`yarn.federation.machine-list` | `<path of macihne-list file>` | Path of machine-list
file used by `SubClusterResolver`. Each line of the file is a node with sub-cluster and rack
information. Below is the example: <br/> <br/> node1, subcluster1, rack1 <br/>
node2, subcluster2, rack1 <br/> node3, subcluster3, rack2 <br/> node4, subcluster3,
rack2 |
 
 ###ON RMs:
 
@@ -226,8 +226,7 @@ These are extra configurations that should appear in the **conf/yarn-site.xml**
 
 | Property | Example | Description |
 |:---- |:---- |
-|`yarn.resourcemanager.epoch` | `<unique-epoch>` | The seed value for the epoch. This
is used to guarantee uniqueness of container-IDs generate by different RMs. It must therefore
be unique among sub-clusters and `well-spaced` to allow for failures which increment epoch.
Increments of 1000 allow for a large number of sub-clusters and
- practically ensure near-zero chance of collisions (a clash will only happen if a container
is still alive for 1000 restarts of one RM, while the next RM never restarted, and an app
requests more containers). |
+|`yarn.resourcemanager.epoch` | `<unique-epoch>` | The seed value for the epoch. This
is used to guarantee uniqueness of container-IDs generate by different RMs. It must therefore
be unique among sub-clusters and `well-spaced` to allow for failures which increment epoch.
Increments of 1000 allow for a large number of sub-clusters and practically ensure near-zero
chance of collisions (a clash will only happen if a container is still alive for 1000 restarts
of one RM, while the next RM never restarted, and an app requests more containers). |
 
 Optional:
 
@@ -285,7 +284,7 @@ In order to submit jobs to a Federation cluster one must create a seperate
set o
 | Property | Example | Description |
 |:--- |:--- |
 | `yarn.resourcemanager.address` | `<router_host>:8050` | Redirects jobs launched at
the client to the router's client RM port. |
-| `yarn.resourcemanger.scheduler.address` | `localhost:8049` | Redirects jobs to the federation
AMRMProxy port.|
+| `yarn.resourcemanager.scheduler.address` | `localhost:8049` | Redirects jobs to the federation
AMRMProxy port.|
 
 Any YARN jobs for the cluster can be submitted from the client configurations described above.
In order to launch a job through federation, first start up all the clusters involved in the
federation as described [here](../../hadoop-project-dist/hadoop-common/ClusterSetup.html).
Next, start up the router on the router machine with the following command:
 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message