Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 30 Jul 2014 14:49:39 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12730741.1406714450331.75564.1406731779120@arcas>
In-Reply-To: <JIRA.12730741.1406714450331@arcas>
References: <JIRA.12730741.1406714450331@arcas>
Subject: [jira] [Commented] (HDFS-6782) Improve FS editlog logSync
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079326#comment-14079326 ] 

Hadoop QA commented on HDFS-6782:
---------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12658643/HDFS-6782.001.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version 2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

                  org.apache.hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean
                  org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
                  org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
                  org.apache.hadoop.hdfs.TestDFSRename
                  org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
                  org.apache.hadoop.hdfs.TestDatanodeConfig
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
                  org.apache.hadoop.hdfs.TestDatanodeBlockScanner
                  org.apache.hadoop.hdfs.server.namenode.TestXAttrConfigFlag
                  org.apache.hadoop.hdfs.TestBlockReaderFactory
                  org.apache.hadoop.hdfs.TestDFSClientRetries
                  org.apache.hadoop.hdfs.server.namenode.TestFSImage
                  org.apache.hadoop.hdfs.TestFSOutputSummer
                  org.apache.hadoop.cli.TestCacheAdminCLI
                  org.apache.hadoop.hdfs.TestFileCreationDelete
                  org.apache.hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
                  org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
                  org.apache.hadoop.hdfs.TestSnapshotCommands
                  org.apache.hadoop.hdfs.TestFileConcurrentReader
                  org.apache.hadoop.tools.TestJMXGet
                  org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
                  org.apache.hadoop.hdfs.server.datanode.TestDiskError
                  org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
                  org.apache.hadoop.hdfs.TestFileAppend2
                  org.apache.hadoop.fs.viewfs.TestViewFsWithXAttrs
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
                  org.apache.hadoop.hdfs.web.TestWebHdfsWithAuthenticationFilter
                  org.apache.hadoop.fs.TestUrlStreamHandler
                  org.apache.hadoop.hdfs.server.namenode.TestMetaSave
                  org.apache.hadoop.hdfs.TestDFSRemove
                  org.apache.hadoop.hdfs.TestLeaseRecovery
                  org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
                  org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractSeek
                  org.apache.hadoop.hdfs.server.datanode.TestHSync
                  org.apache.hadoop.fs.TestFcHdfsPermission
                  org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
                  org.apache.hadoop.security.TestPermission
                  org.apache.hadoop.hdfs.TestReplication
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
                  org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
                  org.apache.hadoop.hdfs.server.namenode.TestAclConfigFlag
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
                  org.apache.hadoop.hdfs.TestParallelShortCircuitReadUnCached
                  org.apache.hadoop.hdfs.TestPipelines
                  org.apache.hadoop.hdfs.TestFileCreationClient
                  org.apache.hadoop.hdfs.TestParallelShortCircuitRead
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
                  org.apache.hadoop.cli.TestAclCLI
                  org.apache.hadoop.hdfs.server.namenode.TestSequentialBlockId
                  org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
                  org.apache.hadoop.hdfs.TestRenameWhileOpen
                  org.apache.hadoop.hdfs.TestBlocksScheduledCounter
                  org.apache.hadoop.hdfs.TestRollingUpgradeRollback
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractDelete
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
                  org.apache.hadoop.hdfs.server.namenode.TestFSImageWithXAttr
                  org.apache.hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
                  org.apache.hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
                  org.apache.hadoop.fs.viewfs.TestViewFsFileStatusHdfs
                  org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
                  org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
                  org.apache.hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
                  org.apache.hadoop.hdfs.server.namenode.TestHostsFiles
                  org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes
                  org.apache.hadoop.hdfs.tools.TestDFSAdminWithHA
                  org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
                  org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
                  org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
                  org.apache.hadoop.hdfs.TestDataTransferKeepalive
                  org.apache.hadoop.hdfs.TestFSInputChecker
                  org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
                  org.apache.hadoop.fs.TestResolveHdfsSymlink
                  org.apache.hadoop.hdfs.TestDFSUpgrade
                  org.apache.hadoop.hdfs.server.namenode.TestLeaseManager
                  org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
                  org.apache.hadoop.fs.permission.TestStickyBit
                  org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
                  org.apache.hadoop.hdfs.server.balancer.TestBalancer
                  org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
                  org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
                  org.apache.hadoop.hdfs.TestRead
                  org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
                  org.apache.hadoop.hdfs.TestDistributedFileSystem
                  org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
                  org.apache.hadoop.hdfs.TestHDFSTrash
                  org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
                  org.apache.hadoop.hdfs.web.TestHttpsFileSystem
                  org.apache.hadoop.hdfs.server.datanode.TestStorageReport
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractOpen
                  org.apache.hadoop.hdfs.TestDatanodeRegistration
                  org.apache.hadoop.hdfs.TestCrcCorruption
                  org.apache.hadoop.fs.shell.TestHdfsTextCommand
                  org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl
                  org.apache.hadoop.hdfs.TestSmallBlock
                  org.apache.hadoop.hdfs.TestDFSRollback
                  org.apache.hadoop.hdfs.server.namenode.TestAuditLogger
                  org.apache.hadoop.hdfs.server.datanode.TestTransferRbw
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean
                  org.apache.hadoop.hdfs.server.namenode.ha.TestHAMetrics
                  org.apache.hadoop.hdfs.server.namenode.TestFsck
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractCreate
                  org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
                  org.apache.hadoop.net.TestNetworkTopology
                  org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
                  org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
                  org.apache.hadoop.hdfs.web.TestWebHdfsTokens
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
                  org.apache.hadoop.hdfs.TestAbandonBlock
                  org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
                  org.apache.hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
                  org.apache.hadoop.hdfs.TestConnCache
                  org.apache.hadoop.fs.TestFcHdfsSetUMask
                  org.apache.hadoop.hdfs.TestFileCorruption
                  org.apache.hadoop.hdfs.TestHDFSServerPorts
                  org.apache.hadoop.hdfs.server.namenode.TestFileLimit
                  org.apache.hadoop.hdfs.TestFsShellPermission
                  org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
                  org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
                  org.apache.hadoop.hdfs.TestFileAppend4
                  org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade
                  org.apache.hadoop.hdfs.TestDFSFinalize
                  org.apache.hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
                  org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
                  org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
                  org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
                  org.apache.hadoop.hdfs.TestLocalDFS
                  org.apache.hadoop.hdfs.server.datanode.TestCachingStrategy
                  org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
                  org.apache.hadoop.hdfs.server.datanode.TestReadOnlySharedStorage
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
                  org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
                  org.apache.hadoop.fs.TestSymlinkHdfsDisable
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeInitStorage
                  org.apache.hadoop.hdfs.TestBlockMissingException
                  org.apache.hadoop.hdfs.TestFileAppend3
                  org.apache.hadoop.hdfs.TestFileStatus
                  org.apache.hadoop.hdfs.TestQuota
                  org.apache.hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
                  org.apache.hadoop.hdfs.TestIsMethodSupported
                  org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
                  org.apache.hadoop.hdfs.TestListFilesInDFS
                  org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
                  org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
                  org.apache.hadoop.hdfs.server.namenode.TestFSDirectory
                  org.apache.hadoop.TestGenericRefresh
                  org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
                  org.apache.hadoop.hdfs.server.namenode.TestCreateEditsLog
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
                  org.apache.hadoop.hdfs.web.TestWebHDFSXAttr
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractConcat
                  org.apache.hadoop.hdfs.web.TestWebHDFSAcl
                  org.apache.hadoop.hdfs.TestDataTransferProtocol
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
                  org.apache.hadoop.hdfs.TestRollingUpgrade
                  org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
                  org.apache.hadoop.hdfs.server.namenode.TestStartup
                  org.apache.hadoop.hdfs.TestBalancerBandwidth
                  org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner
                  org.apache.hadoop.hdfs.TestPersistBlocks
                  org.apache.hadoop.security.TestPermissionSymlinks
                  org.apache.hadoop.hdfs.TestDFSAddressConfig
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
                  org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
                  org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
                  org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
                  org.apache.hadoop.fs.viewfs.TestViewFsHdfs
                  org.apache.hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
                  org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
                  org.apache.hadoop.hdfs.web.TestWebHDFS
                  org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
                  org.apache.hadoop.hdfs.web.TestWebHDFSForHA
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
                  org.apache.hadoop.hdfs.TestDFSPermission
                  org.apache.hadoop.hdfs.TestFileCreation
                  org.apache.hadoop.fs.viewfs.TestViewFsAtHdfsRoot
                  org.apache.hadoop.fs.TestEnhancedByteBufferAccess
                  org.apache.hadoop.hdfs.server.datanode.TestIncrementalBrVariations
                  org.apache.hadoop.hdfs.server.namenode.TestAddBlockRetry
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend
                  org.apache.hadoop.hdfs.TestFileAppend
                  org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
                  org.apache.hadoop.fs.TestSymlinkHdfsFileContext
                  org.apache.hadoop.hdfs.server.namenode.TestFSNamesystemMBean
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
                  org.apache.hadoop.hdfs.web.TestWebHdfsContentLength
                  org.apache.hadoop.hdfs.TestMiniDFSCluster
                  org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
                  org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
                  org.apache.hadoop.hdfs.TestParallelRead
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
                  org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
                  org.apache.hadoop.hdfs.TestWriteRead
                  org.apache.hadoop.hdfs.TestPread
                  org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
                  org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
                  org.apache.hadoop.TestRefreshCallQueue
                  org.apache.hadoop.hdfs.TestLeaseRecovery2
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
                  org.apache.hadoop.hdfs.TestRestartDFS
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
                  org.apache.hadoop.hdfs.TestGetFileChecksum
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot
                  org.apache.hadoop.security.TestRefreshUserMappings
                  org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache
                  org.apache.hadoop.hdfs.server.datanode.TestIncrementalBlockReports
                  org.apache.hadoop.hdfs.TestDFSMkdirs
                  org.apache.hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
                  org.apache.hadoop.fs.TestFcHdfsCreateMkdir
                  org.apache.hadoop.hdfs.TestClientReportBadBlock
                  org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
                  org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
                  org.apache.hadoop.hdfs.TestHFlush
                  org.apache.hadoop.hdfs.TestParallelUnixDomainRead
                  org.apache.hadoop.hdfs.TestDatanodeReport
                  org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser
                  org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
                  org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
                  org.apache.hadoop.hdfs.TestDFSShellGenericOptions
                  org.apache.hadoop.cli.TestXAttrCLI
                  org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
                  org.apache.hadoop.hdfs.TestFileAppendRestart
                  org.apache.hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
                  org.apache.hadoop.hdfs.TestSetrepIncreasing
                  org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
                  org.apache.hadoop.hdfs.server.namenode.TestAuditLogs
                  org.apache.hadoop.hdfs.server.namenode.TestINodeFile
                  org.apache.hadoop.hdfs.security.TestDelegationToken
                  org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
                  org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
                  org.apache.hadoop.hdfs.TestReadWhileWriting
                  org.apache.hadoop.hdfs.TestDFSClientFailover
                  org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
                  org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
                  org.apache.hadoop.hdfs.TestLease
                  org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
                  org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl
                  org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
                  org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing
                  org.apache.hadoop.hdfs.TestModTime
                  org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
                  org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
                  org.apache.hadoop.hdfs.TestDFSOutputStream
                  org.apache.hadoop.hdfs.server.namenode.TestNameNodeRpcServer
                  org.apache.hadoop.hdfs.TestEncryptedTransfer
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRename
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
                  org.apache.hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing
                  org.apache.hadoop.hdfs.TestFetchImage
                  org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
                  org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
                  org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
                  org.apache.hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
                  org.apache.hadoop.hdfs.TestClientBlockVerification
                  org.apache.hadoop.fs.viewfs.TestViewFileSystemWithXAttrs
                  org.apache.hadoop.hdfs.TestGetBlocks
                  org.apache.hadoop.hdfs.server.namenode.TestFileContextXAttr
                  org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
                  org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
                  org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication
                  org.apache.hadoop.hdfs.server.namenode.TestDeleteRace
                  org.apache.hadoop.hdfs.TestSetrepDecreasing
                  org.apache.hadoop.cli.TestHDFSCLI
                  org.apache.hadoop.hdfs.TestMissingBlocksAlert
                  org.apache.hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN
                  org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
                  org.apache.hadoop.hdfs.security.token.block.TestBlockToken
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
                  org.apache.hadoop.fs.contract.hdfs.TestHDFSContractMkdir
                  org.apache.hadoop.hdfs.TestDFSShell
                  org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
                  org.apache.hadoop.hdfs.qjournal.server.TestJournalNode
                  org.apache.hadoop.hdfs.TestDecommission
                  org.apache.hadoop.hdfs.server.namenode.TestMetadataVersionOutput
                  org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit
                  org.apache.hadoop.hdfs.TestBlockReaderLocal
                  org.apache.hadoop.hdfs.server.namenode.TestNameNodeXAttr
                  org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
                  org.apache.hadoop.hdfs.TestHDFSFileSystemContract
                  org.apache.hadoop.hdfs.TestDFSStartupVersions
                  org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
                  org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl
                  org.apache.hadoop.fs.TestHDFSFileContextMainOperations
                  org.apache.hadoop.fs.viewfs.TestViewFsDefaultValue
                  org.apache.hadoop.hdfs.TestDatanodeDeath
                  org.apache.hadoop.hdfs.TestFileCreationEmpty
                  org.apache.hadoop.hdfs.TestAppendDifferentChecksum
                  org.apache.hadoop.hdfs.server.namenode.TestBackupNode
                  org.apache.hadoop.hdfs.TestSafeMode
                  org.apache.hadoop.hdfs.TestListFilesInFileContext
                  org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
                  org.apache.hadoop.hdfs.TestSeekBug
                  org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
                  org.apache.hadoop.hdfs.TestDisableConnCache
                  org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
                  org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs
                  org.apache.hadoop.hdfs.server.namenode.TestAddBlock
                  org.apache.hadoop.hdfs.TestLargeBlock
                  org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy
                  org.apache.hadoop.hdfs.TestMultiThreadedHflush
                  org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
                  org.apache.hadoop.hdfs.TestClose
                  org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
                  org.apache.hadoop.hdfs.TestSetTimes
                  org.apache.hadoop.fs.TestGlobPaths
                  org.apache.hadoop.fs.viewfs.TestViewFsWithAcls
                  org.apache.hadoop.fs.viewfs.TestViewFileSystemWithAcls
                  org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7497//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7497//console

This message is automatically generated.

> Improve FS editlog logSync
> --------------------------
>
>                 Key: HDFS-6782
>                 URL: https://issues.apache.org/jira/browse/HDFS-6782
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>         Attachments: HDFS-6782.001.patch
>
>
> In NN, it uses a double buffer (bufCurrent, bufReady) for log sync, bufCurrent it to buffer new coming edit ops and bufReady is for flushing. This's efficient. When flush is ongoing, and bufCurrent is full, NN goes to force log sync, and all new Ops are blocked (since force log sync is protected by FSNameSystem write lock). After the flush finished, the new Ops are still blocked, but actually at this time, bufCurrent is free and Ops can go ahead and write to the buffer. The following diagram shows the detail. This JIRA is for this improvement.  Thanks [~umamaheswararao] for confirming this issue.
> {code}
> edit1(txid1) ------ write to bufCurrent -------- logSync --------- (swap buffer)flushing -------
> edit2(txid2) ------ write to bufCurrent -------- logSync --------- waiting -------
> edit3(txid3) ------ write to bufCurrent -------- logSync --------- waiting -------
> edit4(txid4) ------ write to bufCurrent -------- logSync --------- waiting -------
> edit5(txid5) ------ write to bufCurrent --full-- force sync --------- waiting -------
> edit6(txid6) ------ blocked
> ...
> editn(txidn) ------ blocked
> {code}
> After the flush, it becomes
> {code}
> edit1(txid1) ------ write to bufCurrent -------- logSync --------- finished --------
> edit2(txid2) ------ write to bufCurrent -------- logSync --------- flushing -------
> edit3(txid3) ------ write to bufCurrent -------- logSync --------- waiting -------
> edit4(txid4) ------ write to bufCurrent -------- logSync --------- waiting -------
> edit5(txid5) ------ write to bufCurrent --full-- force sync --------- waiting -------
> edit6(txid6) ------ blocked
> ...
> editn(txidn) ------ blocked
> {code}
> After edit1 finished, bufCurrent is free, and the thread which flushes txid2 will also flushes txid3-5, so we should return from the force sync of edit5 and FSNamesystem write lock will be freed (Don't worry that edit5 Op will return, since there will be a normal logSync after the force logSync and there will wait for sync finished). This is the idea of this JIRA. 


--
This message was sent by Atlassian JIRA
(v6.2#6252)