hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3605) Missing Block in following scenario
Date Tue, 10 Jul 2012 06:09:36 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410077#comment-13410077
] 

Uma Maheswara Rao G commented on HDFS-3605:
-------------------------------------------

{code}
public void processQueuedMessagesForBlock(Block b) throws IOException {
    Queue<ReportedBlockInfo> queue = pendingDNMessages.takeBlockQueue(b);
    if (queue == null) {
      // Nothing to re-process
      return;
    }
    processQueuedMessages(queue);
  }
{code}

I think here, on first OP_CLOSE edit processing it is trying to process the QueuedMessagesForBlock.
But here queued messages may contains the more future block as well, because duw to many append
calls, SNN queued that messages.
Instead processing all the queued messages for that block, it is make sense to process that
current block(current OP_CLOSE genstamp block)?

pendingDNMessages.takeBlockQueue(b); will give set of blocks which are matching to the blockID,
because it was queuse by block ID. did not consider genstamp. But, by considering current
case, do we need to consider getstamp also, while getting the blcok from queued message and
process. Because there moght be some more OPs which will come with that block as part of furtheer
appends. At that time, anyway that respective genstamp blocks will get processed right?
 


                
> Missing Block in following scenario
> -----------------------------------
>
>                 Key: HDFS-3605
>                 URL: https://issues.apache.org/jira/browse/HDFS-3605
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 2.0.1-alpha
>            Reporter: Brahma Reddy Battula
>            Assignee: Todd Lipcon
>         Attachments: TestAppendBlockMiss.java
>
>
> Open file for append
> Write data and sync.
> After next log roll and editlog tailing in standbyNN close the append stream.
> Call append multiple times on the same file, before next editlog roll.
> Now abruptly kill the current active namenode.
> Here block is missed..
> this may be because of All latest blocks were queued in StandBy Namenode. 
> During failover, first OP_CLOSE was processing the pending queue and adding the block
to corrupted block. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message