hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mungeol Heo <mungeol....@gmail.com>
Subject Re: mismatch corrupt blocks from fsck and dfadmin report.
Date Tue, 27 Dec 2016 08:47:15 GMT
In my case, the "Blocks with corrupt replicas", which from "hdfs dfsadmin
-report" command, became 0 after a while.

On Tue, Dec 27, 2016 at 4:15 PM, Mungeol Heo <mungeol.heo@gmail.com> wrote:

> +1
>
> On Tue, Jan 12, 2016 at 7:20 PM, 권병창 <magnum.c@navercorp.com> wrote:
>
>> Hi. hadooper!
>>
>>
>>
>> I use hadoop-2.7.1 and  my cluster has 130 nodes.
>>
>>
>>
>> recently I am facing a problem.
>>
>>
>>
>> I have found corrupt block by nagios.
>>
>>
>>
>> nagios request http://namenode01:50070/jmx?qry=Hadoop:service=NameN
>> ode,name=FSNamesystem.
>>
>>
>>
>> below is result.  notice CorruptBlocks is 1.
>>
>>  {
>>
>>   "beans" : [ {
>>     "name" : "Hadoop:service=NameNode,name=FSNamesystem",
>>     "modelerType" : "FSNamesystem",
>>     "tag.Context" : "dfs",
>>     "tag.HAState" : "active",
>>     "tag.Hostname" : "css0700.nhnsystem.com",
>>     "MissingBlocks" : 0,
>>     "MissingReplOneBlocks" : 0,
>>     "ExpiredHeartbeats" : 10,
>>     "TransactionsSinceLastCheckpoint" : 820630,
>>     "TransactionsSinceLastLogRoll" : 1916,
>>     "LastWrittenTransactionId" : 376685578,
>>     "LastCheckpointTime" : 1452583950883,
>>     "CapacityTotal" : 1650893130075660,
>>     "CapacityTotalGB" : 1537514.0,
>>     "CapacityUsed" : 1237079990848257,
>>     "CapacityUsedGB" : 1152121.0,
>>     "CapacityRemaining" : 410189981364473,
>>     "CapacityRemainingGB" : 382019.0,
>>     "CapacityUsedNonDFS" : 3623157862930,
>>     "TotalLoad" : 6717,
>>     "SnapshottableDirectories" : 0,
>>     "Snapshots" : 0,
>>     "BlocksTotal" : 4034155,
>>     "FilesTotal" : 2866690,
>>     "PendingReplicationBlocks" : 0,
>>     "UnderReplicatedBlocks" : 0,
>>     "CorruptBlocks" : 1,
>>     "ScheduledReplicationBlocks" : 0,
>>     "PendingDeletionBlocks" : 0,
>>     "ExcessBlocks" : 87,
>>     "PostponedMisreplicatedBlocks" : 0,
>>     "PendingDataNodeMessageCount" : 0,
>>     "MillisSinceLastLoadedEdits" : 0,
>>     "BlockCapacity" : 67108864,
>>     "StaleDataNodes" : 0,
>>     "TotalFiles" : 2866690
>>   } ]
>> }
>>
>>
>>
>>
>>
>> however  'hdfs fsck / -list-corruptfileblocks' does not found.
>>
>>
>>
>> below is fsck result.
>>
>>  The filesystem under path '/' has 0 CORRUPT files
>>
>>
>>
>> 'hdfs dfsadmin -report'  is similar first result.
>>
>>  Configured Capacity: 1650892318461452 (1.47 PB)
>>
>> Present Capacity: 1647353181258422 (1.46 PB)
>>
>> DFS Remaining: 408711410865856 (371.72 TB)
>>
>> DFS Used: 1238641770392566 (1.10 PB)
>>
>> DFS Used%: 75.19%
>>
>> Under replicated blocks: 0
>>
>> Blocks with corrupt replicas: 1
>>
>> Missing blocks: 0
>>
>> Missing blocks (with replication factor 1): 0
>>
>>
>>
>> My question is
>>
>>
>>
>> 1. why different result?
>>
>> 2. How do I find corrupt filename? I wonder which file is corrupt.
>>
>>
>>
>> Thank you.
>>
>>
>>
>
>

Mime
View raw message