drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tshi...@apache.org
Subject [04/25] drill git commit: rip out MapR stuff
Date Tue, 12 May 2015 05:56:46 GMT
rip out MapR stuff


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/913b998b
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/913b998b
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/913b998b

Branch: refs/heads/gh-pages
Commit: 913b998b531ee73fcb05084e55ec84381d4b4d3b
Parents: 80bbe06
Author: Kristine Hahn <khahn@maprtech.com>
Authored: Thu May 7 16:06:15 2015 -0700
Committer: Kristine Hahn <khahn@maprtech.com>
Committed: Thu May 7 16:06:15 2015 -0700

----------------------------------------------------------------------
 .../050-configuring-multitenant-resources.md    | 49 ++------------------
 1 file changed, 5 insertions(+), 44 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/913b998b/_docs/configure-drill/050-configuring-multitenant-resources.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/050-configuring-multitenant-resources.md b/_docs/configure-drill/050-configuring-multitenant-resources.md
index f8ca673..2acd0e7 100644
--- a/_docs/configure-drill/050-configuring-multitenant-resources.md
+++ b/_docs/configure-drill/050-configuring-multitenant-resources.md
@@ -2,25 +2,9 @@
 title: "Configuring Multitenant Resources"
 parent: "Configuring a Multitenant Cluster"
 ---
-Drill operations are memory and CPU-intensive. Currently, Drill resources are managed outside
of any cluster management service, such as the MapR warden service. In a multitenant or any
other type of cluster, YARN-enabled or not, you configure memory and memory usage limits for
Drill by modifying drill-env.sh as described in ["Configuring Drill Memory"]({{site.baseurl}}/docs/configuring-drill-memory).
+Drill operations are memory and CPU-intensive. Currently, Drill resources are managed outside
of any cluster management service. In a multitenant or any other type of cluster, YARN-enabled
or not, you configure memory and memory usage limits for Drill by modifying drill-env.sh as
described in ["Configuring Drill Memory"]({{site.baseurl}}/docs/configuring-drill-memory).
 
-Configure a multitenant cluster to account for resources required for Drill. For example,
on a MapR cluster, ensure warden accounts for resources required for Drill. Configuring `drill-env.sh`
allocates resources for Drill to use during query execution, while configuring the following
properties in `warden-drill-bits.conf` prevents warden from committing the resources to other
processes.
-
-    service.heapsize.min=<some value in MB>
-    service.heapsize.max=<some value in MB>
-    service.heapsize.percent=<a whole number>
-
-{% include startimportant.html %}Set the `service.heapsize` properties in `warden.drill-bits.conf`
regardless of whether you changed defaults in `drill-env.sh` or not.{% include endimportant.html
%}
-
-The section, ["Configuring Drill in a YARN-enabled MapR Cluster"]({{site.baseurl}}/docs/configuring-multitenant-resources#configuring-drill-in-a-yarn-enabled-mapr-cluster)
shows an example of setting the `service.heapsize` properties. The `service.heapsize.percent`
is the percentage of memory for the service bounded by minimum and maximum values. Typically,
users change `service.heapsize.percent`. Using a percentage has the advantage of increasing
or decreasing resources according to different node's configuration. For more information
about the `service.heapsize` properties, see the section, ["warden.<servicename>.conf"](http://doc.mapr.com/display/MapR/warden.%3Cservicename%3E.conf).
-
-You need to statically partition the cluster to designate which partition handles which workload.
To configure resources for Drill in a MapR cluster, modify one or more of the following files
in `/opt/mapr/conf/conf.d` that the installation process creates. 
-
-* `warden.drill-bits.conf`
-* `warden.nodemanager.conf`
-* `warden.resourcemanager.conf`
-
-Configure Drill memory by modifying `warden.drill-bits.conf` in YARN and non-YARN clusters.
Configure other resources by modifying `warden.nodemanager.conf `and `warden.resourcemanager.conf
`in a YARN-enabled cluster.
+Configure a multitenant cluster manager to account for resources required for Drill. Configuring
`drill-env.sh` allocates resources for Drill to use during query execution. It might be necessary
to configure the cluster manager from committing the resources to other processes.
 
 ## Configuring Drill in a YARN-enabled MapR Cluster
 
@@ -49,34 +33,11 @@ ResourceManager and NodeManager memory in `warden.resourcemanager.conf`
and
     service.heapsize.max=325
     service.heapsize.percent=2
 
-Change these settings for NodeManager and ResourceManager to reconfigure the total memory
required for YARN services to run. If you want to place an upper limit on memory set YARN_NODEMANAGER_HEAPSIZE
or YARN_RESOURCEMANAGER_HEAPSIZE environment variable in `/opt/mapr/hadoop/hadoop-2.5.1/etc/hadoop/yarn-env.sh`.
The `-Xmx` option is not set, allowing memory on to grow as needed.
-
-### MapReduce v1 Resources
-
-The following default settings in `/opt/mapr/conf/warden.conf` control MapReduce v1 memory:
-
-    mr1.memory.percent=50
-    mr1.cpu.percent=50
-    mr1.disk.percent=50
-
-Modify these settings to reconfigure MapReduce v1 resources to suit your application needs,
as described in section ["Resource Allocation for Jobs and Applications"](http://doc.mapr.com/display/MapR/Resource+Allocation+for+Jobs+and+Applications)
of the MapR documentation. Remaining memory is given to YARN applications. 
-
-
-### MapReduce v2 and other Resources
-
-You configure memory for each service by setting three values in `warden.conf`.
-
-    service.command.<servicename>.heapsize.percent
-    service.command.<servicename>.heapsize.max
-    service.command.<servicename>.heapsize.min
-
-Configure memory for other services in the same manner, as described in [MapR documentation](http://doc.mapr.com/display/MapR/warden.%3Cservicename%3E.conf)
+Change these settings for NodeManager and ResourceManager to reconfigure the total memory
required for YARN services to run. If you want to place an upper limit on memory set YARN_NODEMANAGER_HEAPSIZE
or YARN_RESOURCEMANAGER_HEAPSIZE environment variable. Do not set the `-Xmx` option to allow
the heap to grow as needed.
 
-For more information about managing memory in a MapR cluster, see the following sections
in the MapR documentation:
+### MapReduce Resources
 
-* [Memory Allocation for Nodes](http://doc.mapr.com/display/MapR/Memory+Allocation+for+Nodes)
 
-* [Cluster Resource Allocation](http://doc.mapr.com/display/MapR/Cluster+Resource+Allocation)
 
-* [Customizing Memory Settings for MapReduce v1](http://doc.mapr.com/display/MapR/Customize+Memory+Settings+for+MapReduce+v1)
 
+Modify MapReduce memory to suit your application needs. Remaining memory is typically given
to YARN applications. 
 
 ## How to Manage Drill CPU Resources
 Currently, you do not manage CPU resources within Drill. [Use Linux `cgroups`](http://en.wikipedia.org/wiki/Cgroups)
to manage the CPU resources.
\ No newline at end of file


Mime
View raw message