hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1497) Write pipeline sequence numbers should be sequential with no skips or duplicates
Date Tue, 07 Dec 2010 18:09:09 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968853#action_12968853

Todd Lipcon commented on HDFS-1497:

Unfortunately it's not quite that simple, since you have to consider the multi-threaded case.
For example, if you have two threads call hflush() at the same time, the first will set {{lastFlushedOffset}}
in its synchronized section, after sending the data into the pipeline. It then drops the lock
to wait for the ack. If a second thread then calls hflush() it _also_ needs to wait for the
same ack - it can't just return, otherwise it's acting like data has been acknowledged that
is still somewhere in the pipeline.

> Write pipeline sequence numbers should be sequential with no skips or duplicates
> --------------------------------------------------------------------------------
>                 Key: HDFS-1497
>                 URL: https://issues.apache.org/jira/browse/HDFS-1497
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 0.20-append, 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-1497.txt, hdfs-1497.txt, hdfs-1497.txt
> In HDFS-895 we discovered that multiple hflush() calls in a row without intervening writes
could cause a skip in sequence number. This doesn't seem to have any direct consequences,
but we should maintain and assert the invariant that sequence numbers have no gaps or duplicates.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message