hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4274) BlockPoolSliceScanner does not close verification log during shutdown
Date Wed, 12 Dec 2012 00:51:21 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth updated HDFS-4274:
--------------------------------

    Attachment: HDFS-4274.2.patch

Here is the new version of the patch:

# During {{DataNode}} shutdown, change {{DataBlockScanner#run}} to shut down every allocated
{{BlockPoolSliceScanner}}, which in turn closes the verification logs.  This part is the same
as the earlier patch.
# {{DataBlockScanner#shutdown}} also joins to the background blockScannerThread.  There is
a slightly tricky bit in here about avoiding a deadlock.  I explained it in comments.  This
is necessary to make shutdown synchronous, so that the caller knows deterministically that
shutdown has completed.  In the case of {{MiniDFSCluster}}, it means that it is safe to delete
the working directories, because nothing will be holding locks.
# Change {{BlockPoolSliceScanner#scan}} to use {{Thread#isInterrupted}} instead of {{Thread#interrupted}}.
 {{BlockPoolSliceScanner#scan}} runs within the blockScannerThread, which depends on seeing
the interrupted status of the thread to know when to exit.  (See the while loop condition
in {{DataBlockScanner#run}}.)  When {{BlockPoolSliceScanner#scan}} was calling {{Thread#interrupted}},
it would clear the current thread's interrupted status, so that the while loop in {{DataBlockScanner#run}}
would never see it, and therefore would never exit.  This was occasionally causing tests to
enter an infinite loop on Windows.

With this patch in place, I am consistently seeing good test results on Mac and Windows, with
no tests blocked on a prior {{MiniDFSCluster}} holding a lock on a verification log.
                
> BlockPoolSliceScanner does not close verification log during shutdown
> ---------------------------------------------------------------------
>
>                 Key: HDFS-4274
>                 URL: https://issues.apache.org/jira/browse/HDFS-4274
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 3.0.0, trunk-win
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HDFS-4274.1.patch, HDFS-4274.2.patch
>
>
> {{BlockPoolSliceScanner}} holds open a handle to a verification log.  This file is not
getting closed during process shutdown.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message