hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6768) Improve performance of yarn api record toString and fromString
Date Wed, 19 Jul 2017 21:25:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093803#comment-16093803

Jason Lowe commented on YARN-6768:

Thanks for updating the patch!  Nice to see the state fall away and things get simpler as
a result.

The toString paths look great, just a few nits on the conversion back from a String.

Nit: It is less work to do indexOf('_') than indexOf("_")

Rather than searching for the first underscore, creating a substring starting at 0 to that
position, and then comparing that substring to a known prefix we can avoid some object creation
and a redundant prefix scan by leveraging startsWith and the length of the known prefix. 
Similarly we can use String#regionMatches to avoid a substring creation for the epoch string.
 For example:
    int pos1 = CONTAINER_PREFIX.length();
    if (!containerIdStr.startsWith(CONTAINER_PREFIX) || containerIdStr.charAt(pos1) != '_')
      throw new IllegalArgumentException("Invalid ContainerId prefix: "
          + containerIdStr.substring(0, pos1 + 1));
    int pos2 = containerIdStr.indexOf('_', pos1 + 1);
    String clusterTimestamp;
    if (containerIdStr.regionMatches(pos1 + 1, EPOCH_PREFIX, 0, EPOCH_PREFIX.length()) {
      String epochStr = containerId.substring(pos1 + 1 + EPOCH_PREFIX.length(), pos2);
      epoch = Integer.parseInt(epochStr);

The use of startsWith to avoid the initial substring applies to app attempt and app ID parsing
as well.  The test gets even cheaper if we bother to precompute a String that is the prefix
with the underscore already attached because we wouldn't need to do the extra check for the
'_' character.

This patch also causes some of the TestConverterUtils tests to fail in hadoop-yarn-common.
 Arguably some of these conversion tests should be moved/adapted to hadoop-yarn-api so they
are closer to the code in question.

> Improve performance of yarn api record toString and fromString
> --------------------------------------------------------------
>                 Key: YARN-6768
>                 URL: https://issues.apache.org/jira/browse/YARN-6768
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Jonathan Eagles
>            Assignee: Jonathan Eagles
>         Attachments: YARN-6768.1.patch, YARN-6768.2.patch, YARN-6768.3.patch, YARN-6768.4.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message