hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12396) Webhdfs file system should get delegation token from kms provider.
Date Mon, 04 Dec 2017 15:33:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276947#comment-16276947
] 

Rushabh S Shah commented on HDFS-12396:
---------------------------------------

The javac warning in branch-2 and branch-2.8 patch is unrelated to my patch.
Many of the failed tests in branch-2 and branch-2.8 are due to {{java.lang.OutOfMemoryError:
unable to create new native thread}}.


Re-reun of the tests that failed in branch-2.8

{noformat}
-------------------------------------------------------
 T E S T S
-------------------------------------------------------

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.688 sec - in org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.964 sec - in org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDFSMkdirs
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.205 sec - in org.apache.hadoop.hdfs.TestDFSMkdirs
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDFSOutputStream
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.214 sec - in org.apache.hadoop.hdfs.TestDFSOutputStream
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDFSShell
Tests run: 47, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.771 sec - in org.apache.hadoop.hdfs.TestDFSShell
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDistributedFileSystem
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 57.417 sec - in org.apache.hadoop.hdfs.TestDistributedFileSystem
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestFileAppendRestart
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.681 sec - in org.apache.hadoop.hdfs.TestFileAppendRestart
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestLeaseRecovery2
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.039 sec - in org.apache.hadoop.hdfs.TestLeaseRecovery2
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestReplaceDatanodeFailureReplication
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 60.845 sec - in org.apache.hadoop.hdfs.TestReplaceDatanodeFailureReplication
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHDFS


Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.016 sec - in org.apache.hadoop.hdfs.web.TestWebHDFS
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHDFSForHA
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.36 sec - in org.apache.hadoop.hdfs.web.TestWebHDFSForHA
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.438 sec - in org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.385 sec - in org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHDFSXAttr
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.906 sec - in org.apache.hadoop.hdfs.web.TestWebHDFSXAttr

Results :

Tests run: 177, Failures: 0, Errors: 0, Skipped: 0
{noformat}



Re-reun of the tests that failed in branch-2
{noformat}
-------------------------------------------------------
 T E S T S
-------------------------------------------------------

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.crypto.TestHdfsCryptoStreams
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.457 sec - in org.apache.hadoop.hdfs.crypto.TestHdfsCryptoStreams
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestAppendSnapshotTruncate
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.724 sec - in org.apache.hadoop.hdfs.TestAppendSnapshotTruncate
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestClose
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.693 sec - in org.apache.hadoop.hdfs.TestClose
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.91 sec - in org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDFSRename
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.901 sec - in org.apache.hadoop.hdfs.TestDFSRename
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDFSShell
Tests run: 47, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.197 sec - in org.apache.hadoop.hdfs.TestDFSShell
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.561 sec - in org.apache.hadoop.hdfs.TestDFSUpgrade
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestEncryptedTransfer
Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 144.369 sec - in org.apache.hadoop.hdfs.TestEncryptedTransfer
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestFileAppend
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.753 sec - in org.apache.hadoop.hdfs.TestFileAppend
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestHdfsAdmin
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.594 sec - in org.apache.hadoop.hdfs.TestHdfsAdmin
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestHDFSServerPorts
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.647 sec - in org.apache.hadoop.hdfs.TestHDFSServerPorts
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestHFlush
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.168 sec - in org.apache.hadoop.hdfs.TestHFlush
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestLease
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.258 sec - in org.apache.hadoop.hdfs.TestLease
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestMaintenanceState
Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 242.794 sec - in org.apache.hadoop.hdfs.TestMaintenanceState
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestRenameWhileOpen
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.288 sec - in org.apache.hadoop.hdfs.TestRenameWhileOpen
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.148 sec - in org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestSetrepDecreasing
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.414 sec - in org.apache.hadoop.hdfs.TestSetrepDecreasing
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestTrashWithSecureEncryptionZones
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.04 sec - in org.apache.hadoop.hdfs.TestTrashWithSecureEncryptionZones
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.53 sec - in org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHDFS
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 126.487 sec - in org.apache.hadoop.hdfs.web.TestWebHDFS
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.961 sec - in org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.537 sec - in org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed
in 8.0
Running org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.346 sec - in org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
{noformat}


Diff of the last patch (#008) from the previously reviewed patch (#005).
{noformat}
> @@ -0,0 +1,190 @@
412d414
< +@InterfaceStability.Evolving
414c416
< +public class HdfsKMSUtil {
---
> +public final class HdfsKMSUtil {
418a421,422
> +  private HdfsKMSUtil() { /* Hidden constructor */ }
> +
556c560
< index 34f5d6ecfe1..c62318b9e06 100644
{noformat}
Removed {{@InterfaceStability.Evolving}} since {{@InterfaceStability.Unstable}} is already
present.
Added final to class {{HdfsKMSUtil}} and added private constructor to {{HdfsKMSUtil}} to make
checkstyle happy.

[~xiaochen], [~daryn]: mind reviewing one more time ?
Final set of patches: {{HDFS-12396.008.patch, HDFS-12396-branch-2.002.patch, HDFS-12396-branch-2.8.002.patch}}

> Webhdfs file system should get delegation token from kms provider.
> ------------------------------------------------------------------
>
>                 Key: HDFS-12396
>                 URL: https://issues.apache.org/jira/browse/HDFS-12396
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: encryption, kms, webhdfs
>            Reporter: Rushabh S Shah
>            Assignee: Rushabh S Shah
>         Attachments: HDFS-12396-branch-2.001.patch, HDFS-12396-branch-2.002.patch, HDFS-12396-branch-2.8.001.patch,
HDFS-12396-branch-2.8.002.patch, HDFS-12396-branch-2.8.patch, HDFS-12396-branch-2.patch, HDFS-12396.001.patch,
HDFS-12396.002.patch, HDFS-12396.003.patch, HDFS-12396.004.patch, HDFS-12396.005.patch, HDFS-12396.006.patch,
HDFS-12396.007.patch, HDFS-12396.008.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message