hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4178) [storage implementation] app id as string can cause incorrect ordering
Date Thu, 17 Sep 2015 20:21:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804423#comment-14804423

Vrushali C commented on YARN-4178:

In hRaven, we started with storing hadoop job ids as a tuple of JT/RM start time and the sequence
number, exactly for this reason: to maintain the right ordering. 

But this is good as long as the prefix for app ids is "application_". If something changes
and we have a different prefix, then querying older data (older format row keys) becomes harder.

Column name ordering may not be an issue, I think.

For row keys, where do we see this incorrect ordering in row keys? In the applications table?
But I think there is a prefix or "user!cluster!flow! flow runid! " to each row key before
the application id, no? 

> [storage implementation] app id as string can cause incorrect ordering
> ----------------------------------------------------------------------
>                 Key: YARN-4178
>                 URL: https://issues.apache.org/jira/browse/YARN-4178
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
> Currently the app id is used in various places as part of row keys and in column names.
However, they are treated as strings for the most part. This will cause a problem with ordering
when the id portion of the app id rolls over to the next digit.
> For example, "app_1234567890_100" will be considered *earlier* than "app_1234567890_99".
We should correct this.

This message was sent by Atlassian JIRA

View raw message