hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6634) inotify in HDFS
Date Thu, 28 Aug 2014 22:39:10 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114511#comment-14114511

Andrew Wang commented on HDFS-6634:

Hey James, nice work here. This is all pretty much cosmetic stuff, so +1 pending.

* not a fan of the labelled break, consider redoing it somehow?

* You could turn some of the double slash comments on the private variables into javadocs.
* Related to that, since we're cancelling the Callable on a timeout, is the cached thread
pool necessary? I know that cancellation could be delayed for various reasons, but a fixed
size thread pool would be more conservative.
* boolean notifyMissingEvents is not used
* poll, you could pull out {{e.getCause()}} into a variable once, it's used a bunch.
* For {{take()}}, let's make it randomly choose a timeout within the doubling window, to avoid
stampedes. A slow NN could lead to the clients synchronizing their sleeps and then all arriving
at the same time.
* Regarding the lower bound, we could move that constant into take since it's not used elsewhere.
* I'd like to upper bound the take() sleep, at say 2 mins. We don't want this going up forever.
* Optional, but if you wanted to slap down some DEBUG logging, that'd also be nice for debugging.
slf4j is preferred.

* "Garbage collected" isn't quite the right term, "checkpointed" would be more accurate, i.e.
"cleaned up as part of checkpointing."

Various EditLogs:
* Typically we put @Override on its own line

* Could make these double-slash comments into javadoc too

* Lower the LOG.error to warn? it's recoverable.

* Would be good to call getEventsBehindEstimate() just to have some sort of test coverage

> inotify in HDFS
> ---------------
>                 Key: HDFS-6634
>                 URL: https://issues.apache.org/jira/browse/HDFS-6634
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: hdfs-client, namenode, qjm
>            Reporter: James Thomas
>            Assignee: James Thomas
>         Attachments: HDFS-6634.2.patch, HDFS-6634.3.patch, HDFS-6634.4.patch, HDFS-6634.5.patch,
HDFS-6634.6.patch, HDFS-6634.7.patch, HDFS-6634.patch, inotify-design.2.pdf, inotify-design.3.pdf,
inotify-design.4.pdf, inotify-design.pdf, inotify-intro.2.pdf, inotify-intro.pdf
> Design a mechanism for applications like search engines to access the HDFS edit stream.

This message was sent by Atlassian JIRA

View raw message