hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4178) [storage implementation] app id as string in row keys can cause incorrect ordering
Date Tue, 29 Sep 2015 01:11:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934442#comment-14934442

Sangjin Lee commented on YARN-4178:

Thanks [~varun_saxena] for the proposed patch. I'm generally in agreement with the approach.

As for the "application" prefix, is there a discussion in the YARN community in general of
making this prefix customizable? I'm just not familiar with the current state of that discussion,
although I recall hearing about it in the past. That would help us determine whether this
is significant enough for us to care about it.

One small nit on the patch: in {{TimelineReaderUtils.java}}, there is following code:

    long clusterTs = Long.MAX_VALUE - Bytes.toLong(appIdBytes, 0);
    int seqId = Integer.MAX_VALUE - Bytes.toInt(appIdBytes, Bytes.SIZEOF_LONG);

Although it is exactly equivalent, for readability it might be bit better to spell out the
size and the offset explicitly, and also use {{invert()}}:

    long clusterTs = invert(Bytes.toLong(appIdBytes, 0, Bytes.SIZEOF_LONG));
    int seqId = invert(Bytes.toInt(appIdBytes, Bytes.SIZEOF_LONG, Bytes.SIZEOF_INT));

That would also make it a little more symmetric with the writer util. Also, use {{invert()}}
for the writer util.

> [storage implementation] app id as string in row keys can cause incorrect ordering
> ----------------------------------------------------------------------------------
>                 Key: YARN-4178
>                 URL: https://issues.apache.org/jira/browse/YARN-4178
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-4178-YARN-2928.01.patch
> Currently the app id is used in various places as part of row keys. However, currently
they are treated as strings. This will cause a problem with ordering when the id portion of
the app id rolls over to the next digit.
> For example, "app_1234567890_10000" will be considered *earlier* than "app_1234567890_9999".
We should correct this.

This message was sent by Atlassian JIRA

View raw message