hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5281) Explore supporting a simpler back-end implementation for ATS v2
Date Tue, 21 Jun 2016 18:50:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342439#comment-15342439

Joep Rottinghuis commented on YARN-5281:

[~kasha] raised a good point to focus on ensuring that we eliminate as many entry barriers
as possible. I understand the concern about having to support a separate HBase installation
for small Hadoop clusters. It isn't ideal if people have to become an HBase expert before
using Yarn. I can open a separate jira to ship a dead-simple configuration where all needed
processes for HBase can be launched on a single machine (perhaps by default where the user
chooses to run the RM process). This would probably depend on YARN-5045.

As [~varun_saxena] pointed out, we did start with trying to maintain both HBase and file-based
implementation. As the features progressed and became more sophisticated it became increasingly
more difficult to maintain feature parity, which is why we ultimately decided to move the
file-based implementation to be for testing only.

[~ozawa] highlighted the desire to have a simple implementation on HDFS. As I imagine what
that would look like, especially to serve reads, filters and queries one would end up with
what would essentially be a mini-HBase implementation.

We've separately discussed a potential solution to spool writes to disk for if/when HBase
is temporarily unavailable. We can see if that can be used to serve some use-cases to test
out the API for writes.

> Explore supporting a simpler back-end implementation for ATS v2
> ---------------------------------------------------------------
>                 Key: YARN-5281
>                 URL: https://issues.apache.org/jira/browse/YARN-5281
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Joep Rottinghuis
> During the merge discussion [~kasha] raised the question whether we would support simpler
backend for users to try out, in addition to the HBase implementation.
> The understanding is that this would not be meant to scale, but it could simplify initial
adoption and early usage.
> I'm filing this jira to gather the merits and challenges of such approach in one place.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message