ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Wagle (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-13684) Ambari Metrics Collector failed to start in Ambari 2.1.2 with UnicodeEncodeError
Date Mon, 02 Nov 2015 23:00:28 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986218#comment-14986218
] 

Siddharth Wagle commented on AMBARI-13684:
------------------------------------------

The problem is with special characters in the KAFKA service metrics file. We use this file
as a part of stack_advisor calculations for AMS split points in 2.1.2

Following is a grep on non ASCII characters that reveals the problem.

{noformat}
root@os-u12-upgrade-2ga-latest-4:~# grep --color='auto' -P -n "[\x80-\xFF]" /var/lib/ambari-server/resources/common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/KAFKA.txt
43:���kafka.network.RequestMetrics.RequestsPerSec.request.OffsetFetch.count���
45:���kafka.network.RequestMetrics.RequestsPerSec.request.OffsetCommit.count
47:kafka.network.RequestMetrics.RequestsPerSec.request.LeaderAndIsr.1MinuteRate���
49:���kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.meanRate���
51:���kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.5MinuteRate���
52:kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.1MinuteRate���
53:kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.15MinuteRate���
55:���kafka.network.RequestMetrics.RequestsPerSec.request.Heartbeat.count
56:���kafka.network.RequestMetrics.RequestsPerSec.request.Heartbeat.5MinuteRate
57:���kafka.network.RequestMetrics.RequestsPerSec.request.Heartbeat.1MinuteRate���
58:kafka.network.RequestMetrics.RequestsPerSec.request.Heartbeat.15MinuteRate���
60:kafka.network.RequestMetrics.RequestsPerSec.request.FetchFollower.5MinuteRate���
62:���kafka.network.RequestMetrics.RequestsPerSec.request.FetchFollower.15MinuteRate
63:���kafka.network.RequestMetrics.RequestsPerSec.request.FetchConsumer.meanRate
64:���kafka.network.RequestMetrics.RequestsPerSec.request.FetchConsumer.count���
66:���kafka.network.RequestMetrics.RequestsPerSec.request.FetchConsumer.1MinuteRate
67:���kafka.network.RequestMetrics.RequestsPerSec.request.Fetch.count
{noformat}

Other service metrics are ok.

*Why is this not always reproducible*:
- Split point calculation picks up 50 equidistant points for each application to use for the
final split points result.
- We have about 17 points that are bad out of 190, so we have a 4.47 % chance of picking a
bad point.
- User needs to have Kafka installed

*Workaround*:
- Call API to get site configuration for ams:
{code}
http://172.22.64.164:8080/api/v1/clusters/cl1/configurations?type=ams-site
{code}
Pick latest
- Check the value of _timeline.metrics.cluster.aggregate.splitpoints_ and _timeline.metrics.host.aggregate.splitpoints_
- Look for special non-ascii characters example:
{code}
"dfs.datanode.ReplaceBlockOpAvgTime,kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.1MinuteRate
,master.Master.ProcessCallTime_num_ops,regionserver.Server.blockCacheEvictionCount"
{code}
Check _1MinuteRate_ in the example above.
- Use _/var/lib/ambari-server/resources/scripts/configs.sh_ to modify the value and remove
the special character and restart the service.

> Ambari Metrics Collector failed to start in Ambari 2.1.2 with UnicodeEncodeError
> --------------------------------------------------------------------------------
>
>                 Key: AMBARI-13684
>                 URL: https://issues.apache.org/jira/browse/AMBARI-13684
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-metrics, ambari-server
>    Affects Versions: 2.1.2
>            Reporter: Siddharth Wagle
>            Assignee: Siddharth Wagle
>            Priority: Critical
>             Fix For: 2.1.3
>
>
> Stderr
> {noformat}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
line 131, in <module>
>     AmsCollector().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 216, in execute
>     method(env)
>   File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
line 44, in start
>     self.configure(env, action = 'start') # for security
>   File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
line 41, in configure
>     ams(name='collector')
>   File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89,
in thunk
>     return fn(*args, **kwargs)
>   File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py",
line 202, in ams
>     group=params.user_group
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154,
in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
152, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
118, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/xml_config.py",
line 67, in action_create
>     encoding = self.resource.encoding
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154,
in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
152, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
118, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 90, in action_create
>     content = self._get_content()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 127, in _get_content
>     return content()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 51,
in __call__
>     return self.get_content()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 142,
in get_content
>     rendered = self.template.render(self.context)
>   File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in
render
>     return self.environment.handle_exception(exc_info, True)
>   File "<template>", line 5, in top-level template code
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u2028' in position 108: ordinal
not in range(128)
> {noformat}
> |Old Ambari|NEW Ambari|HDP Stack
> |Ambari 1.7.0|Ambari2.1.2|2.2.0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message