hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6243) HA NameNode transition to active or shutdown may leave lingering image transfer thread.
Date Mon, 14 Apr 2014 23:23:17 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth updated HDFS-6243:
--------------------------------

    Attachment: HDFS-6243.2.patch

Thanks for looking at the patch, Jing.  Here is v2.  I've added an assertion at the end of
the test that {{FSImage#getMostRecentCheckpointTxId}} returns 0 for the former active.  The
only checkpoint issued in the test is being canceled, so we expect this to remain 0.

I've also added one more change in {{StandbyCheckpointer}}.  I noticed that the background
thread might still spend a fair amount of time blocked in {{wait}} inside the throttler. 
We can make it break out of the throttle faster by calling {{Future#cancel}} to interrupt
the underlying thread.

> HA NameNode transition to active or shutdown may leave lingering image transfer thread.
> ---------------------------------------------------------------------------------------
>
>                 Key: HDFS-6243
>                 URL: https://issues.apache.org/jira/browse/HDFS-6243
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: ha, namenode
>    Affects Versions: 3.0.0, 2.4.0
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HDFS-6243.1.patch, HDFS-6243.2.patch
>
>
> With HDFS-4816, the standby uploads the new checkpoint to the active in a background
thread.  This thread may continue to execute after transitioning to active or shutting down.
 This is marginally wasteful of bandwidth if the image transfer is large, and it also breaks
subsequent tests on Windows since the thread continues to hold a lock on a file descriptor
in the storage directory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message