hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Bartolomaeus (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5012) replica.getGenerationStamp() may be >= recoveryId
Date Wed, 01 Jun 2016 12:15:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310206#comment-15310206
] 

Christian Bartolomaeus commented on HDFS-5012:
----------------------------------------------

Some additional information to the above stack trace: The error happened after a machine running
a DataNode crashed and rebooted unexpectedly. The warning message was logged by that machine
after the reboot when the DataNode process started. 

On three other DataNodes (those holding replicas of the block in question) the following error
was logged:

{noformat}
PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException: THIS IS NOT SUPPOSED
TO HAPPEN: replica.getGenerationStamp() >= recoveryId = 1527175689, block=blk_2570851709037266390_1527175689,
replica=FinalizedReplica, blk_2570851709037266390_1527175689, FINALIZED
  getNumBytes()     = 48360562
  getBytesOnDisk()  = 48360562
  getVisibleLength()= 48360562
  getVolume()       = /var/lib/hdfs5/data/current
  getBlockFile()    = /var/lib/hdfs5/data/current/BP-655596758-10.10.34.1-1341996058045/current/finalized/subdir38/subdir48/blk_2570851709037266390
  unlinked          =false
{noformat}

> replica.getGenerationStamp() may be >= recoveryId
> -------------------------------------------------
>
>                 Key: HDFS-5012
>                 URL: https://issues.apache.org/jira/browse/HDFS-5012
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.5-alpha
>            Reporter: Ted Yu
>         Attachments: testReplicationQueueFailover.txt
>
>
> The following was first observed by [~jdcryans] in TestReplicationQueueFailover running
against 2.0.5-alpha:
> {code}
> 2013-07-16 17:14:33,340 ERROR [IPC Server handler 7 on 35081] security.UserGroupInformation(1481):
PriviledgedActionException as:ec2-user (auth:SIMPLE) cause:java.io.IOException: THIS IS NOT
SUPPOSED TO HAPPEN: replica.getGenerationStamp() >= recoveryId = 1041, block=blk_4297992342878601848_1041,
replica=FinalizedReplica, blk_4297992342878601848_1041, FINALIZED
>   getNumBytes()     = 794
>   getBytesOnDisk()  = 794
>   getVisibleLength()= 794
>   getVolume()       = /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current
>   getBlockFile()    = /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current/BP-1477359609-10.197.55.49-1373994849464/current/finalized/blk_4297992342878601848
>   unlinked          =false
> 2013-07-16 17:14:33,341 WARN  [org.apache.hadoop.hdfs.server.datanode.DataNode$2@64a1fcba]
datanode.DataNode(1894): Failed to obtain replica info for block (=BP-1477359609-10.197.55.49-1373994849464:blk_4297992342878601848_1041)
from datanode (=127.0.0.1:47006)
> java.io.IOException: THIS IS NOT SUPPOSED TO HAPPEN: replica.getGenerationStamp() >=
recoveryId = 1041, block=blk_4297992342878601848_1041, replica=FinalizedReplica, blk_4297992342878601848_1041,
FINALIZED
>   getNumBytes()     = 794
>   getBytesOnDisk()  = 794
>   getVisibleLength()= 794
>   getVolume()       = /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current
>   getBlockFile()    = /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current/BP-1477359609-10.197.55.49-1373994849464/current/finalized/blk_4297992342878601848
>   unlinked          =false
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message