hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7446) HDFS inotify should have the ability to determine what txid it has read up to
Date Wed, 26 Nov 2014 02:19:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225605#comment-14225605
] 

Andrew Wang commented on HDFS-7446:
-----------------------------------

Hey Colin, thanks for working on this. You definitely bring up a good point about the txids.
Since this is marked as unstable and still quite new, I think it's okay to make sweeping changes
to the API.

I had just a few high-level review comments, the code itself looks fine:

* It feels like we have a mismatch between the underlying data and our objects. The need for
the VHS-rewind in getTxidBatchSize is one example, what we really want there is an iterator
of EditEvents, with one EditEvents per txid (name is just a suggestion).
* The txid could also be moved into EditEvents which would also save some bytes.

I'm hoping this isn't too bad to do, since the edit log translator already returns an Event[]
per op, and it seems like most of the PB code can be reused.

> HDFS inotify should have the ability to determine what txid it has read up to
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-7446
>                 URL: https://issues.apache.org/jira/browse/HDFS-7446
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: dfsclient
>    Affects Versions: 2.6.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-7446.001.patch
>
>
> HDFS inotify should have the ability to determine what txid it has read up to.  This
will allow users who want to avoid missing any events to record this txid and use it to resume
reading events at the spot they left off.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message