hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5265) Make HBase configuration for the timeline service configurable
Date Thu, 16 Jun 2016 20:16:05 GMT

     [ https://issues.apache.org/jira/browse/YARN-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joep Rottinghuis updated YARN-5265:
-----------------------------------
    Attachment: ATS v2 cluster deployment v1.png

Attaching diagram that [~vrushalic] created. This helps in visualizing the challenge. The
cluster on which the nodemanagers run and/or connect to by default do not necessarily have
to be the same as the HBase cluster that ATS needs to write to.

There are two configurations involved. The Yarn configuration for the source cluster, that
is the cluster where the timelineservice client run on (as part of the nodemanager, resourcemanager,
or reader service). This configuration will have most of the Yarn ATS properties.
The second configuration is the one that identifies the HBase cluster to which the timelineservice
client has to write to (or read from). These settings may not be the same as the default cluster
that tasks on the cluster connect to.

It isn't good enough to require the ATS HBase cluster configs to be on the classpath of the
nodemanagers, resource manager etc.

> Make HBase configuration for the timeline service configurable
> --------------------------------------------------------------
>
>                 Key: YARN-5265
>                 URL: https://issues.apache.org/jira/browse/YARN-5265
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Joep Rottinghuis
>            Assignee: Joep Rottinghuis
>         Attachments: ATS v2 cluster deployment v1.png
>
>
> Currently we create "default" HBase configurations, this works as long as the user places
the appropriate configuration on the classpath.
> This works fine for a standalone Hadoop cluster.
> However, if a user wants to monitor an HBase cluster and has a separate ATS HBase cluster,
then it can become tricky to create the right classpath for the nodemanagers and still have
tasks have their separate configs.
> It will be much easier to add a yarn configuration to let cluster admins configure which
HBase to connect to to write ATS metrics to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message