hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Esteban Gutierrez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17799) HBCK region boundaries check can return false negatives when IOExceptions are thrown
Date Tue, 21 Mar 2017 22:28:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935467#comment-15935467
] 

Esteban Gutierrez commented on HBASE-17799:
-------------------------------------------

The new output is something like this when running the the {{-boundaries}} flag:

{code}
2017-03-21 14:00:29,051 INFO  [main] util.HBaseFsck: Starting region boundaries check. It
might take a while...
2017-03-21 14:00:29,059 INFO  [main] util.HBaseFsck: Scanning 6 (online) regions for integrity
of store files and META boundaries.
2017-03-21 14:00:29,064 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-03-21 14:00:29,111 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
ERROR: Region boundaries mismatch in TestTable,00000000000000000000004000,1490031498711.4bc692a1f072260b0d9bf298c2af555f.
and store file:/var/folders/br/gq9645xd17s6v7xgjsmcd2dr0000gp/T/hbase-esteban/hbase/data/default/TestTable/4bc692a1f072260b0d9bf298c2af555f/info/57894cc7de074b16a0dd0bada4d508ea
2017-03-21 14:00:29,112 WARN  [main] util.HBaseFsck: file:/var/folders/br/gq9645xd17s6v7xgjsmcd2dr0000gp/T/hbase-esteban/hbase/data/default/TestTable/4bc692a1f072260b0d9bf298c2af555f/info/57894cc7de074b16a0dd0bada4d508ea
is not within boundaries: store: [00000000000000000000004000 -> 00000000000000000000011999)region:
[00000000000000000000004000 -> 00000000000000000000008000)
2017-03-21 14:00:29,112 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-03-21 14:00:29,113 WARN  [main] util.HBaseFsck: Skipping file:/var/folders/br/gq9645xd17s6v7xgjsmcd2dr0000gp/T/hbase-esteban/hbase/data/default/TestTable/4bc692a1f072260b0d9bf298c2af555f/info/empty
from region TestTable,00000000000000000000004000,1490031498711.4bc692a1f072260b0d9bf298c2af555f.
got an exception.
org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from
file file:/var/folders/br/gq9645xd17s6v7xgjsmcd2dr0000gp/T/hbase-esteban/hbase/data/default/TestTable/4bc692a1f072260b0d9bf298c2af555f/info/empty
        at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:474)
        at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:517)
        at org.apache.hadoop.hbase.util.HBaseFsck.checkRegionBoundaries(HBaseFsck.java:706)
        at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:625)
        at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:4433)
        at org.apache.hadoop.hbase.util.HBaseFsck$HBaseFsckTool.run(HBaseFsck.java:4236)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:4224)
Caused by: java.lang.IllegalArgumentException
        at java.nio.Buffer.position(Buffer.java:244)
        at org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:395)
        at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:459)
        ... 8 more
2017-03-21 14:00:29,116 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-03-21 14:00:29,117 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-03-21 14:00:29,118 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-03-21 14:00:29,120 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2017-03-21 14:00:29,121 INFO  [main] util.HBaseFsck: Region boundaries test scanned: 6 files
in 6 regions.
{code}

If there is an IOException on a file we log the exception now, and we skip the file and then
we move to the next file. If we find an HFileLink we skip it now.


> HBCK region boundaries check can return false negatives when IOExceptions are thrown
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-17799
>                 URL: https://issues.apache.org/jira/browse/HBASE-17799
>             Project: HBase
>          Issue Type: Bug
>          Components: hbck
>    Affects Versions: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>            Reporter: Esteban Gutierrez
>            Assignee: Esteban Gutierrez
>         Attachments: HBASE-17799.master.001.patch
>
>
> When enabled, HBaseFsck#checkRegionBoundaries will crawl all HFiles across all namespaces
and tables when {{-boundaries}} is specified. However if an IOException is thrown by accessing
a corrupt HFile, an un-handled HLink or by any other reason, we will only log the exception
and stop crawling the HFiles and potentially reporting the wrong result.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message