hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8809) HDFS fsck reports file corruption (Missing blocks) when HBase is running, with Hadoop 2.7.1 and HBase 1.1.1
Date Thu, 23 Jul 2015 01:35:04 GMT
Matt Foley created HDFS-8809:
--------------------------------

             Summary: HDFS fsck reports file corruption (Missing blocks) when HBase is running,
with Hadoop 2.7.1 and HBase 1.1.1
                 Key: HDFS-8809
                 URL: https://issues.apache.org/jira/browse/HDFS-8809
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: tools
    Affects Versions: 2.7.0
         Environment: Hadoop 2.7.1 and HBase 1.1.1, on SUSE11sp3 (other Linuxes not tested,
probably not platform-dependent).  This did NOT happen with Hadoop 2.4 and HBase 0.98.
            Reporter: Matt Foley


Whenever HBase is running, the "hdfs fsck /"  reports four hbase-related files in the path
"hbase/data/WALs/" as CORRUPT. Even after letting the cluster sit idle for a couple hours,
it is still in the corrupt state.  If HBase is shut down, the problem goes away.  If HBase
is then restarted, the problem recurs.

{code}
hades1:/var/opt/teradata/packages # su hdfs
hdfs@hades1:/var/opt/teradata/packages> hdfs fsck /
Connecting to namenode via http://hades1.labs.teradata.com:50070/fsck?ugi=hdfs&path=%2F
FSCK started by hdfs (auth:SIMPLE) from /39.0.8.2 for path / at Wed Jun 24 20:40:17 GMT 2015
...
/apps/hbase/data/WALs/hades4.labs.teradata.com,16020,1435168292684/hades4.labs.teradata.com%2C16020%2C1435168292684.default.1435175500556:
MISSING 1 blocks of total size 83 B.
/apps/hbase/data/WALs/hades5.labs.teradata.com,16020,1435168290466/hades5.labs.teradata.com%2C16020%2C1435168290466..meta.1435175562144.meta:
MISSING 1 blocks of total size 83 B.
/apps/hbase/data/WALs/hades5.labs.teradata.com,16020,1435168290466/hades5.labs.teradata.com%2C16020%2C1435168290466.default.1435175498500:
MISSING 1 blocks of total size 83 B.
/apps/hbase/data/WALs/hades6.labs.teradata.com,16020,1435168292373/hades6.labs.teradata.com%2C16020%2C1435168292373.default.1435175500301:
MISSING 1 blocks of total size 83 B..................................................................................................
....................................................................................................
....................................................................................................
........................................................................................Status:
CORRUPT
 Total size:    723977553 B (Total open files size: 332 B)
 Total dirs:    79
 Total files:   388
 Total symlinks:                0 (Files currently being written: 5)
 Total blocks (validated):      387 (avg. block size 1870743 B) (Total open file blocks (not
validated): 4)
  ********************************
  UNDER MIN REPL'D BLOCKS:      4 (1.0335917 %)
  dfs.namenode.replication.min: 1
  CORRUPT FILES:        4
  MISSING BLOCKS:       4
  MISSING SIZE:         332 B
  ********************************
 Minimally replicated blocks:   387 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Wed Jun 24 20:40:17 GMT 2015 in 7 milliseconds


The filesystem under path '/' is CORRUPT
hdfs@hades1:/var/opt/teradata/packages>
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message