hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7446) HDFS inotify should have the ability to determine what txid it has read up to
Date Mon, 01 Dec 2014 23:11:13 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Colin Patrick McCabe updated HDFS-7446:
---------------------------------------
    Attachment: HDFS-7446.002.patch

This patch refactors things so that we have an {{EventBatch}} structure which gets sent via
the protobuf.  The {{EventBatch}} contains a txid, and all events which pertain to that transaction.
 This eventually makes its way to DFSInotifyEventInputStream, which returns {{EventBatch}}
rather than a single {{Event}}, as in the current code, or an array of events, as in the previous
patch.

As mentioned before, this is an incompatible API change.  It necessary because the current
API gives users no good way to determine that they're "done" with a given transaction ID...
and hence, no way to save their progress and resume there later.

I also renamed {{getEventsBehindEstimate}} to {{getTxidsBehindEstimate}}.  This reflects the
fact that it's measuring transaction IDs, not events.

> HDFS inotify should have the ability to determine what txid it has read up to
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-7446
>                 URL: https://issues.apache.org/jira/browse/HDFS-7446
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: dfsclient
>    Affects Versions: 2.6.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-7446.001.patch, HDFS-7446.002.patch
>
>
> HDFS inotify should have the ability to determine what txid it has read up to.  This
will allow users who want to avoid missing any events to record this txid and use it to resume
reading events at the spot they left off.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message