hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8809) HDFS fsck reports HBase WALs files (under construction) as "CORRUPT" (missing blocks) when HBase is running
Date Thu, 20 Aug 2015 01:10:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704088#comment-14704088
] 

Hadoop QA commented on HDFS-8809:
---------------------------------

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch |  16m 22s | Findbugs (version ) appears to be broken on
trunk. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any @author tags.
|
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to include 1 new
or modified test files. |
| {color:green}+1{color} | javac |   8m 16s | There were no new javac warning messages. |
| {color:green}+1{color} | javadoc |  10m 19s | There were no new javadoc warning messages.
|
| {color:green}+1{color} | release audit |   0m 24s | The applied patch does not increase
the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   0m 34s | There were no new checkstyle issues. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that end in whitespace.
|
| {color:green}+1{color} | install |   1m 42s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 37s | The patch built with eclipse:eclipse.
|
| {color:green}+1{color} | findbugs |   2m 47s | The patch does not introduce any new Findbugs
(version 3.0.0) warnings. |
| {color:green}+1{color} | native |   3m 23s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 177m 26s | Tests failed in hadoop-hdfs. |
| | | 221m 53s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM |
| Timed out tests | org.apache.hadoop.cli.TestHDFSCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | http://issues.apache.org/jira/secure/attachment/12751321/HDFS-8809.000.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 3aac475 |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12048/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12048/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep
3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12048/console |


This message was automatically generated.

> HDFS fsck reports HBase WALs files (under construction) as "CORRUPT" (missing blocks)
when HBase is running
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8809
>                 URL: https://issues.apache.org/jira/browse/HDFS-8809
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 2.7.0
>         Environment: Hadoop 2.7.1 and HBase 1.1.1, on SUSE11sp3 (other Linuxes not tested,
probably not platform-dependent).  This did NOT happen with Hadoop 2.4 and HBase 0.98.
>            Reporter: Sudhir Prakash
>            Assignee: Jing Zhao
>         Attachments: HDFS-8809.000.patch
>
>
> Whenever HBase is running, the "hdfs fsck /"  reports four hbase-related files in the
path "hbase/data/WALs/" as CORRUPT. Even after letting the cluster sit idle for a couple hours,
it is still in the corrupt state.  If HBase is shut down, the problem goes away.  If HBase
is then restarted, the problem recurs.  This was observed with Hadoop 2.7.1 and HBase 1.1.1,
and did NOT happen with Hadoop 2.4 and HBase 0.98.
> {code}
> hades1:/var/opt/teradata/packages # su hdfs
> hdfs@hades1:/var/opt/teradata/packages> hdfs fsck /
> Connecting to namenode via http://hades1.labs.teradata.com:50070/fsck?ugi=hdfs&path=%2F
> FSCK started by hdfs (auth:SIMPLE) from /39.0.8.2 for path / at Wed Jun 24 20:40:17 GMT
2015
> ...
> /apps/hbase/data/WALs/hades4.labs.teradata.com,16020,1435168292684/hades4.labs.teradata.com%2C16020%2C1435168292684.default.1435175500556:
MISSING 1 blocks of total size 83 B.
> /apps/hbase/data/WALs/hades5.labs.teradata.com,16020,1435168290466/hades5.labs.teradata.com%2C16020%2C1435168290466..meta.1435175562144.meta:
MISSING 1 blocks of total size 83 B.
> /apps/hbase/data/WALs/hades5.labs.teradata.com,16020,1435168290466/hades5.labs.teradata.com%2C16020%2C1435168290466.default.1435175498500:
MISSING 1 blocks of total size 83 B.
> /apps/hbase/data/WALs/hades6.labs.teradata.com,16020,1435168292373/hades6.labs.teradata.com%2C16020%2C1435168292373.default.1435175500301:
MISSING 1 blocks of total size 83 B..................................................................................................
> ....................................................................................................
> ....................................................................................................
> ........................................................................................Status:
CORRUPT
>  Total size:    723977553 B (Total open files size: 332 B)
>  Total dirs:    79
>  Total files:   388
>  Total symlinks:                0 (Files currently being written: 5)
>  Total blocks (validated):      387 (avg. block size 1870743 B) (Total open file blocks
(not validated): 4)
>   ********************************
>   UNDER MIN REPL'D BLOCKS:      4 (1.0335917 %)
>   dfs.namenode.replication.min: 1
>   CORRUPT FILES:        4
>   MISSING BLOCKS:       4
>   MISSING SIZE:         332 B
>   ********************************
>  Minimally replicated blocks:   387 (100.0 %)
>  Over-replicated blocks:        0 (0.0 %)
>  Under-replicated blocks:       0 (0.0 %)
>  Mis-replicated blocks:         0 (0.0 %)
>  Default replication factor:    3
>  Average block replication:     3.0
>  Corrupt blocks:                0
>  Missing replicas:              0 (0.0 %)
>  Number of data-nodes:          3
>  Number of racks:               1
> FSCK ended at Wed Jun 24 20:40:17 GMT 2015 in 7 milliseconds
> The filesystem under path '/' is CORRUPT
> hdfs@hades1:/var/opt/teradata/packages>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message