hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tony Reix (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12106) org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS fails on AIX
Date Fri, 19 Jun 2015 14:05:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14593443#comment-14593443
] 

Tony Reix commented on HADOOP-12106:
------------------------------------

Class ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoOutputStream.java
:

public void write(byte[] b, int off, int len) throws IOException {
..........
        encrypt();
      }
    }
  }

private void encrypt() throws IOException {
 .......

    if (encryptor.isContextReset()) {
      /*
       * This code is generally not executed since the encryptor usually
       * maintains encryption context (e.g. the counter) internally. However,
       * some implementations can't maintain context so a re-init is necessary
       * after each encryption call.
       */
      updateEncryptor();
    }
  }

After adding many traces in the code, I've found that, on Ubuntu / i386 / OpenJDK, the updateEncryptor()
method is never called from encrypt.
On AIX, I can see that encryptor.isContextReset() returns true and that updateEncryptor()
is called.

When I comment this      updateEncryptor();  line and rerun tests on AIX, the number of error
fall from :
     Failures: 5, Errors: 1,
to:
    Failures: 3, Errors: 1,

So, it looks to me that IBM JVM is a special case that is not correctly handled by Hadoop
code.

> org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS fails on AIX
> -----------------------------------------------------------------
>
>                 Key: HADOOP-12106
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12106
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.6.0, 2.7.0
>         Environment: Hadoop 2.60 and 2.7+
>  - AIX/PowerPC/IBMJVM
>  - Ubuntu/i386/IBMJVM
>            Reporter: Tony Reix
>         Attachments: mvn.Test.TestCryptoStreamsForLocalFS.res20.AIX.Errors, mvn.Test.TestCryptoStreamsForLocalFS.res20.Ubuntu-i386.IBMJVM.Errors,
mvn.Test.TestCryptoStreamsForLocalFS.res22.OpenJDK.Errors
>
>
> On AIX (IBM JVM available only), many sub-tests of :
>    org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS
> fail:
>  Tests run: 13, Failures: 5, Errors: 1, Skipped: 
>   - testCryptoIV
>   - testSeek
>   - testSkip
>   - testAvailable
>   - testPositionedRead
> When testing SAME exact code on Ubuntu/i386 :
>   - with OpenJDK, all tests are OK
>   - with IBM JVM, tests randomly fail.
> The issue may be in the IBM JVM, or in some Hadoop code that not perfectly handles differences
due to different IBM JVM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message