hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4178) [storage implementation] app id as string can cause incorrect ordering
Date Mon, 21 Sep 2015 17:44:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901057#comment-14901057
] 

Varun Saxena commented on YARN-4178:
------------------------------------

[~jrottinghuis],
No, I did not mean that we can use ApplicationId#toString to create a string which can be
stored in rowkey, if that is what you meant. appid is already in that format.

What I was suggesting was that on the write path, we can store only the cluster timestamp
and sequence number(12 bytes - one long and one int) in the row key and skip storing the "application_"
part. Storing as long and int or 2 longs would ensure correct ordering(although ascending).
So, as you said above Long.MAX_VALUE - X should be used for ensuring descending order.
ApplicationId#toString I was talking in context of read path. On the read path we can read
these 12 bytes from row key and call ApplicationId#newInstance and ApplicationId#toString
to change the timestamp and id to application_ prefix app id in string format, which can then
be sent back to the client. And if prefix changes, ApplicationId will be changed as well(as
it is used all over YARN).

However your comment about storing application_ part in the end to make row key future proof
makes sense. We can go with it.

> [storage implementation] app id as string can cause incorrect ordering
> ----------------------------------------------------------------------
>
>                 Key: YARN-4178
>                 URL: https://issues.apache.org/jira/browse/YARN-4178
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>
> Currently the app id is used in various places as part of row keys and in column names.
However, they are treated as strings for the most part. This will cause a problem with ordering
when the id portion of the app id rolls over to the next digit.
> For example, "app_1234567890_100" will be considered *earlier* than "app_1234567890_99".
We should correct this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message