hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-3486) offlineimageviewer can't read fsimage files that contain persistent delegation tokens
Date Thu, 31 May 2012 21:10:22 GMT
Colin Patrick McCabe created HDFS-3486:
------------------------------------------

             Summary: offlineimageviewer can't read fsimage files that contain persistent
delegation tokens
                 Key: HDFS-3486
                 URL: https://issues.apache.org/jira/browse/HDFS-3486
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Colin Patrick McCabe
            Assignee: Colin Patrick McCabe
            Priority: Minor


OfflineImageViewer (oiv) crashes when trying to read fsimage files that contain persistent
delegation tokens.

Example stack trace:
{code}
Caused by: java.lang.IndexOutOfBoundsException
        at java.io.DataInputStream.readFully(DataInputStream.java:175)
        at org.apache.hadoop.io.Text.readFields(Text.java:284)
        at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.readFields(AbstractDelegationTokenIdentifier.java:178)
        at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processDelegationTokens(ImageLoaderCurrent.java:222)
        at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.loadImage(ImageLoaderCurrent.java:186)
        at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.go(OfflineImageViewer.java:129)
{code}

The oiv and loadFSImage code paths are separate.  The issue here seems to be that the loadFSImage
code path has diverged from the oiv code path.

On the loadFSImage code path (from FSImageFormat#loadCurrentTokens):
{code}
  /**
   * Private helper methods to load Delegation tokens from fsimage
   */
  private synchronized void loadCurrentTokens(DataInputStream in)
      throws IOException {
    int numberOfTokens = in.readInt();
    for (int i = 0; i < numberOfTokens; i++) {
      DelegationTokenIdentifier id = new DelegationTokenIdentifier();
      id.readFields(in);
      long expiryTime = in.readLong();
      addPersistedDelegationToken(id, expiryTime);
    }
  }
{code}
Notice how it loads a 4-byte int after every DelegationTokenIdentifier.

On the oiv code path (from ImageLoaderCurrent#processDelegationTokens):
{code}
    int numDTokens = in.readInt();
    v.visitEnclosingElement(ImageElement.DELEGATION_TOKENS,
        ImageElement.NUM_DELEGATION_TOKENS, numDTokens);
    for(int i=0; i<numDTokens; i++){
      DelegationTokenIdentifier id = new  DelegationTokenIdentifier();
      id.readFields(in);
      v.visit(ImageElement.DELEGATION_TOKEN_IDENTIFIER, id.toString());
    }
{code}

Notice how it does *not* load a 4-byte int after every DelegationTokenIdentifier.

This bug seems to have been introduced by change 916534, the same change which introduced
persistent delegation tokens.  So I don't think oiv was ever able to decode them in the past.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message