ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mildebrandt <ch...@woodenrhino.com>
Subject Re: Overriding values in component configuration using the Ambari UI
Date Tue, 18 Feb 2014 22:04:15 GMT
Hi Jaimin,

Thanks for your patience, I really appreciate it. I still don't entirely
understand how this field in the UI will get populated when I configure the
cluster using the Ambari REST APIs. Since I'm not using the wizard, there
must be a command I'm missing on my side. Can you please provide an example
how I would set this (or point me in a place I can look at the commands the
wizard executes during deployment)?

On a side note, my coworker just told me that he put one value in the text
area (yarn.scheduler.capacity.maximum-applications=10000) and the file on
the hosts was not touched. The other modified parameters in the YARN config
were updated, and the yarn-site.xml file was updated correctly. Any idea
why it wouldn't have updated the capacity-scheduler.xml file?

I'd like to suggest that this field get initially populated with the
default values when creating a cluster through the REST APIs. The reason
being, this file exists on the hosts and is being actively used by the YARN
application....so Ambari should know about it.

I hope everything above makes sense. Please let me know if you need more
details, and thanks again for your continuing help,
-Chris

On Tue, Feb 18, 2014 at 12:47 PM, Jaimin Jetly <jaimin@hortonworks.com>wrote:

> Hi Chris,
>
> Hi Chris,
>
> HDP stack RPM deploys the capacity-scheduler.xml by default at
> /etc/hadoop/conf/capacity-scheduler.xml path. So initially it's the RPM
> and not the REST APIs that create capacity-scheduler.xml file. This file
> is unknown to Ambari until the "capacity-scheduler" configuration resource
> is created with a list of properties by installer wizard via REST APIs.
> Once this is done by the wizard and further yarn service is started by the
> wizard, then the properties listed in the configuration completely
> overrides the RPM deployed file at /etc/hadoop/conf/capacity-scheduler.xml
> path
>
>
> So I suggest insert the same default values in the test area as below
> which are suggested by Ambari-web ui during installation.
>
> yarn.scheduler.capacity.maximum-am-resource-percent=0.2
> yarn.scheduler.capacity.maximum-applications=10000
> yarn.scheduler.capacity.node-locality-delay=40
> yarn.scheduler.capacity.root.acl_administer_queues=*
> yarn.scheduler.capacity.root.capacity=100
> yarn.scheduler.capacity.root.default.acl_administer_jobs=*
> yarn.scheduler.capacity.root.default.acl_submit_jobs=*
> yarn.scheduler.capacity.root.default.capacity=100
> yarn.scheduler.capacity.root.default.maximum-capacity=100
> yarn.scheduler.capacity.root.default.state=RUNNING
> yarn.scheduler.capacity.root.default.user-limit-factor=1
> yarn.scheduler.capacity.root.queues=default
> yarn.scheduler.capacity.root.unfunded.capacity=50
>
>
>
>
>
> On Tue, Feb 18, 2014 at 11:21 AM, Chris Mildebrandt <chris@woodenrhino.com
> > wrote:
>
>> Hi Jaimin,
>>
>> I've never seen this field in any state other than empty. I'll populate
>> it with the key value pairs which match the current file on my system.
>>
>> Is this due to the fact that I create the cluster entirely through the
>> REST APIs? I don't set a value for this using the REST APIs, so I'd assume
>> it would have just been populated with the capacity-scheduler.xml values.
>> Is this a bug that should be opened, or expected behavior?
>>
>> Thanks for the format of this field and for the tip about the shell
>> script to change one property at a time. That helps a lot.
>>
>> -Chris
>>
>>
>> On Tue, Feb 18, 2014 at 11:13 AM, Jaimin Jetly <jaimin@hortonworks.com>wrote:
>>
>>> Hi Chris,
>>>
>>> Normally, The content in the UI "Capacity Scheduler" text area Which
>>> should be list of key=value pairs delimited by newline is reflected in the
>>> /etc/hadoop/conf/capacity-scheduler.xml file. Both of these are always
>>> in sync.
>>>
>>> If you are seeing something similar to below inline image, then Clicking
>>> on undo link will again populate the text area with last saved value:
>>>
>>> [image: Inline image 1]
>>>
>>>
>>> Otherwise the default value as suggested by Ambari for this field is :
>>>
>>> yarn.scheduler.capacity.maximum-am-resource-percent=0.2
>>> yarn.scheduler.capacity.maximum-applications=10000
>>> yarn.scheduler.capacity.node-locality-delay=40
>>> yarn.scheduler.capacity.root.acl_administer_queues=*
>>> yarn.scheduler.capacity.root.capacity=100
>>> yarn.scheduler.capacity.root.default.acl_administer_jobs=*
>>> yarn.scheduler.capacity.root.default.acl_submit_jobs=*
>>> yarn.scheduler.capacity.root.default.capacity=100
>>> yarn.scheduler.capacity.root.default.maximum-capacity=100
>>> yarn.scheduler.capacity.root.default.state=RUNNING
>>> yarn.scheduler.capacity.root.default.user-limit-factor=1
>>> yarn.scheduler.capacity.root.queues=default
>>> yarn.scheduler.capacity.root.unfunded.capacity=50
>>>
>>>
>>> Hope this helps.
>>>
>>>
>>>
>>> On Tue, Feb 18, 2014 at 10:38 AM, Chris Mildebrandt <
>>> chris@woodenrhino.com> wrote:
>>>
>>>> Hi Jaimin,
>>>>
>>>> The graphic didn't come through, but I understand what you're
>>>> explaining. We do have a missing value for the "Capacity Scheduler". How
>>>> should this be normally filled? We have the standard untouched
>>>> /etc/hadoop/conf/capacity-scheduler.xml file on our machines. Should the
>>>> contents of this file normally show up in the UI? Should I copy the
>>>> contents of the file into this text area?
>>>>
>>>> Thanks,
>>>> -Chris
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 10:28 AM, Jaimin Jetly <jaimin@hortonworks.com>wrote:
>>>>
>>>>> Hi Chris,
>>>>>
>>>>> In context to your first email regarding Ambari UI issue,
>>>>> Ambari web-ui disables the save button whenever there is a validation
>>>>> error for any property on a service config page. In that case you should
>>>>> also see the invalidated property highlighted with red color as shown
below
>>>>> in the image. Please expand all the categories and check if any property
is
>>>>> invalidated.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>> If Ambari UI is disabling the button with no validation error of any
>>>>> property on the page then it's a bug and We will need more information
to
>>>>> debug the cause. To start with a screenshot of the yarn config page and
>>>>> javascript console output in case if any JS  error is logged in the console
>>>>> will be helpful.
>>>>>
>>>>> In regards to your second email,
>>>>> The API call that you mentioned should work and it's the correct way
>>>>> to change configs at the cluster level. In addition to directly using
API
>>>>> for configuration changes, you can also use the shell script located
at
>>>>> /var/lib/ambari-server/resources/scripts/configs.sh path on ambari-server
>>>>> host to add/override/delete a single property. The shell script in itself
>>>>> uses the RESTful API in it's implementation but it's really useful while
>>>>> overriding a single config property.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Feb 18, 2014 at 10:09 AM, Chris Mildebrandt <
>>>>> chris@woodenrhino.com> wrote:
>>>>>
>>>>>> Hi Sid,
>>>>>>
>>>>>> I'm actually setting the desired config. Here's an example of the
>>>>>> REST call:
>>>>>>
>>>>>> curl -i -H 'X-Requested-By: xxx' -X PUT -d '{"Clusters":
>>>>>> {"desired_config": {"tag": "custom1", "type": "yarn-site", "properties":
>>>>>> {"yarn.scheduler.minimum-allocation-mb" : "7168"}}}}' --user admin:admin
>>>>>> http://xxx:8080/api/v1/clusters/cl1
>>>>>>
>>>>>> Now, of course, I'd have all my YARN variables in here not just one.
>>>>>> This is just an example to focus on the technique I'm using to change
the
>>>>>> property. This is the property I'm interested in changing. After
running
>>>>>> the above command, I then restart the required components to re-read
the
>>>>>> config.
>>>>>>
>>>>>> Am I doing this incorrectly? Thanks,
>>>>>> -Chris
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 18, 2014 at 9:38 AM, Siddharth Wagle <
>>>>>> swagle@hortonworks.com> wrote:
>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>> Are you using Config groups to overrides configs as per the latest
>>>>>>> API docs ?
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/config-groups.md
>>>>>>>
>>>>>>> It is possible to do this using the Web UI too, using the Manage
>>>>>>> Config Groups link on the Config page.
>>>>>>> Could you please provide the name of the Service and property
that
>>>>>>> you are trying to override ?
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Sid
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Feb 18, 2014 at 9:17 AM, Chris Mildebrandt <
>>>>>>> chris@woodenrhino.com> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I've searched quite a bit, maybe I'm just not using the correct
>>>>>>>> terms. I'm trying to override certain configuration values
for YARN using
>>>>>>>> the Ambari UI. Can someone please give a step-by-step method
to do this?
>>>>>>>> When I try to update the values, the save button stays disabled.
I can get
>>>>>>>> around this by using the REST APIs for now...but it would
be nice to do
>>>>>>>> this in the UI for quick changes.
>>>>>>>>
>>>>>>>> I'm using HDP 2.0.6 with Ambari 1.4.3.38-1.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -Chris
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> CONFIDENTIALITY NOTICE
>>>>>>> NOTICE: This message is intended for the use of the individual
or
>>>>>>> entity to which it is addressed and may contain information that
is
>>>>>>> confidential, privileged and exempt from disclosure under applicable
law.
>>>>>>> If the reader of this message is not the intended recipient,
you are hereby
>>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>>> disclosure or forwarding of this communication is strictly prohibited.
If
>>>>>>> you have received this communication in error, please contact
the sender
>>>>>>> immediately and delete it from your system. Thank You.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -Thanks
>>>>>  Jaimin
>>>>>
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or
>>>>> entity to which it is addressed and may contain information that is
>>>>> confidential, privileged and exempt from disclosure under applicable
law.
>>>>> If the reader of this message is not the intended recipient, you are
hereby
>>>>> notified that any printing, copying, dissemination, distribution,
>>>>> disclosure or forwarding of this communication is strictly prohibited.
If
>>>>> you have received this communication in error, please contact the sender
>>>>> immediately and delete it from your system. Thank You.
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> -Thanks
>>>  Jaimin
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>>
>>
>>
>
>
> --
> -Thanks
>  Jaimin
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
View raw message