Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Mon, 4 May 2015 18:32:07 +0000 (UTC)
From: "Zhijie Shen (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12786796.1427738537000.72604.1430764327065@Atlassian.JIRA>
In-Reply-To: <JIRA.12786796.1427738537000@Atlassian.JIRA>
References: <JIRA.12786796.1427738537000@Atlassian.JIRA>
 <JIRA.12786796.1427738537458@arcas>
Subject: [jira] [Commented] (YARN-3422) relatedentities always return empty
 list when primary filter is set
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527012#comment-14527012 ] 

Zhijie Shen commented on YARN-3422:
-----------------------------------

[~billie.rinaldi@gmail.com], thanks for explaining the rationale. Hence the attache patch should not be the right fix.

bq. In retrospect, the directional nature of the related entity relationship seems to make things more confusing. Perhaps it would be better if relatedness were bidirectional.

I think directional may be okay, but the confusing part is that we're storing A <- B, but we query B -> A, while we always say related entities. In fact, we need to differentiate both. When storing A, B resides in A entity as isRelatedTo entity, and when querying B, A is shown as the relatesTo entity. Of cause, we can querying A, and B should be shown as the isRelatedTo entity, which is not supported here. This problem will be resolved in ATS v2.

Moreover, it's also the limitation about the way we store the primary filter. The index table is a copy of the whole entity (only the information comes with the current put) and attach the primary filter as the prefix of the key. It makes it expensive to define one primary key for an entity, and probably results in different snapshot of the entity with different primary keys. In this example, B doesn't have primary filter C, but even later we add C for B, we will still be not able to get related entity A when querying B via primary filter C. That's one reason why I suggest using reverse index in YARN-3448.

However, for current LeveldbTimelineStore, I'm not sure if we have a quick way to resolve the problem. Thoughts?

> relatedentities always return empty list when primary filter is set
> -------------------------------------------------------------------
>
>                 Key: YARN-3422
>                 URL: https://issues.apache.org/jira/browse/YARN-3422
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: timelineserver
>            Reporter: Chang Li
>            Assignee: Chang Li
>         Attachments: YARN-3422.1.patch
>
>
> When you curl for ats entities with a primary filter, the relatedentities fields always return empty list


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)