hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5431) support cachepool-based limit management in path-based caching
Date Mon, 16 Dec 2013 21:36:08 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13849744#comment-13849744

Andrew Wang commented on HDFS-5431:

Thanks for the reviews, v6 attached. Changes are only to CRM, so you can just review that.

I thought about it, and went way back to the condition variable proposal. I got rid of the
object lock, favoring a new CRM lock with two cvars and some status variables. The synchronization
here is unfortunately still kind of complicated, but I think overall more better.

I also want to note this isn't actually read-after-write consistency since the {{wait}} methods
just wait for the scan count to advance, not for all pending updates to be flushed. I think
this could be fixed (basically, if a scan is ongoing and CRM is dirty, wait for scanCount
to advance by 2 rather than 1), but as this is also a performance tradeoff, I'd really like
to just get this committed and fix it later if necessary.

I also re-examined {{close}} since I was re-doing the synchronization. This snippet looks
like it makes the thread calling {{close}} wait for 60 seconds, where the intent is probably
to interrupt the CRM thread and make it cleanly exit:

    try {
      if (this.isAlive()) {
    } catch (InterruptedException e) {

If you agree, I'll file a follow-on to fix this up. It could affect HA state transitions.

> support cachepool-based limit management in path-based caching
> --------------------------------------------------------------
>                 Key: HDFS-5431
>                 URL: https://issues.apache.org/jira/browse/HDFS-5431
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>    Affects Versions: 3.0.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Andrew Wang
>         Attachments: hdfs-5431-1.patch, hdfs-5431-2.patch, hdfs-5431-3.patch, hdfs-5431-4.patch,
hdfs-5431-5.patch, hdfs-5431-6.patch
> We should support cachepool-based limit management in path-based caching.

This message was sent by Atlassian JIRA

View raw message