hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4274) BlockPoolSliceScanner does not close verification log during shutdown
Date Tue, 11 Dec 2012 18:05:23 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth updated HDFS-4274:

    Status: Open  (was: Patch Available)

Xuan and Ivan, thank you for the code reviews.  I'm going to cancel this patch for now, because
I'm working on an improved version.  I still see a few lingering test failures caused by the
fact that {{DataBlockScanner#shutdown}} performs its shutdown work asynchronous of the caller
within its background blockScannerThread.  I have a patch in progress to make this synchronous,
so that the caller of {{DataBlockScanner#shutdown}} knows deterministically that shutdown
has completed.  I'll let you know when the new patch is available for review.

Ivan posted this question:

Quick question, would it make sense to shutdown BlockPoolSliceScanners on DataBlockScanner#shutdown?
In the current patch you are assuming that it is fine to close the verificationlogs when DataBlockScanner#run
exits what might not be always true (don't know for sure, asking based on the class relationship).
Just a thought, otherwise patch looks good.

I believe this is a safe change.  {{DataNode#shutdown}} calls {{DataBlockScanner#shutdown}}.
 {{DataBlockScanner}} owns the {{BlockPoolSliceScanner}} instances.  All operations on {{BlockPoolSliceScanner}}
that could use the internal {{verificationLog}} instance are initiated from the background
thread of {{DataBlockScanner}}.  Therefore, when the {{DataBlockScanner}} background thread
is exiting, it is safe to close each verification log, because nothing else will attempt to
use it.

> BlockPoolSliceScanner does not close verification log during shutdown
> ---------------------------------------------------------------------
>                 Key: HDFS-4274
>                 URL: https://issues.apache.org/jira/browse/HDFS-4274
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 3.0.0, trunk-win
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HDFS-4274.1.patch
> {{BlockPoolSliceScanner}} holds open a handle to a verification log.  This file is not
getting closed during process shutdown.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message