hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7339) Splitting a hfilelink causes region servers to go down.
Date Wed, 19 Dec 2012 23:49:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536608#comment-13536608
] 

Jonathan Hsieh commented on HBASE-7339:
---------------------------------------

I was able to verify that hfile-links are properly compacted, and that references to hfile
links are compacted. 

Here's some rough evidence:

During a splits you can see refs to hfile links (the two a517dce192074812bc57256f5811a526-2b380883b191a2153df51a1de9e530b4-TestTable.b4fb30c8cac85edbe9d21fbf49c85dbf
files -- one is top the other is bottom)
{code}
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table
-rw-r--r--   3 jon supergroup        323 2012-12-19 14:10 /hbase-0.95/pe-10-table/.tableinfo.0000000001
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/.tmp
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2
-rw-r--r--   3 jon supergroup         43 2012-12-19 14:10 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2/.regioninfo
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2/info
-rw-r--r--   3 jon supergroup         30 2012-12-19 14:10 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2/info/4df6a174e5944429b6e93926fead9b50.b4fb30c8cac85edbe9d21fbf49c85dbf
-rw-r--r--   3 jon supergroup         30 2012-12-19 14:10 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2/info/a517dce192074812bc57256f5811a526-2b380883b191a2153df51a1de9e530b4-TestTable.b4fb30c8cac85edbe9d21fbf49c85dbf
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f
-rw-r--r--   3 jon supergroup         43 2012-12-19 14:10 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/.regioninfo
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/info
-rw-r--r--   3 jon supergroup         30 2012-12-19 14:10 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/info/4df6a174e5944429b6e93926fead9b50.b4fb30c8cac85edbe9d21fbf49c85dbf
-rw-r--r--   3 jon supergroup         30 2012-12-19 14:10 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/info/a517dce192074812bc57256f5811a526-2b380883b191a2153df51a1de9e530b4-TestTable.b4fb30c8cac85edbe9d21fbf49c85dbf
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/b4fb30c8cac85edbe9d21fbf49c85dbf
-rw-r--r--   3 jon supergroup         33 2012-12-19 14:10 /hbase-0.95/pe-10-table/b4fb30c8cac85edbe9d21fbf49c85dbf/.regioninfo
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/b4fb30c8cac85edbe9d21fbf49c85dbf/.splits
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/b4fb30c8cac85edbe9d21fbf49c85dbf/.tmp
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/b4fb30c8cac85edbe9d21fbf49c85dbf/info
-rw-r--r--   3 jon supergroup  387202111 2012-12-19 14:10 /hbase-0.95/pe-10-table/b4fb30c8cac85edbe9d21fbf49c85dbf/info/4df6a174e5944429b6e93926fead9b50
-rw-r--r--   3 jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/b4fb30c8cac85edbe9d21fbf49c85dbf/info/a517dce192074812bc57256f5811a526-2b380883b191a2153df51a1de9e530b4-TestTable
{code}

A cloned table, after compactions and janitor runs.
{code}
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:27 /hbase-0.95/pe-10-table
-rw-r--r--   3 jon supergroup        323 2012-12-19 14:10 /hbase-0.95/pe-10-table/.tableinfo.0000000001
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:10 /hbase-0.95/pe-10-table/.tmp
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:27 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2
-rw-r--r--   3 jon supergroup         43 2012-12-19 14:10 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2/.regioninfo
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:23 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2/info
-rw-r--r--   3 jon supergroup  204366416 2012-12-19 14:23 /hbase-0.95/pe-10-table/016d350dfbc2c9935c4656f0cda954c2/info/c3c6fafa2c6748b5a3763c1c7a031dd2
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:23 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f
-rw-r--r--   3 jon supergroup         43 2012-12-19 14:10 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/.regioninfo
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:23 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/.tmp
drwxr-xr-x   - jon supergroup          0 2012-12-19 14:23 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/info
-rw-r--r--   3 jon supergroup  204812832 2012-12-19 14:23 /hbase-0.95/pe-10-table/598843726920ddee799ede3b02cfd91f/info/2d1d14729312444590733f91c273cf18
{code}
                
> Splitting a hfilelink causes region servers to go down.
> -------------------------------------------------------
>
>                 Key: HBASE-7339
>                 URL: https://issues.apache.org/jira/browse/HBASE-7339
>             Project: HBase
>          Issue Type: Sub-task
>          Components: snapshots
>    Affects Versions: hbase-6055
>            Reporter: Jonathan Hsieh
>            Assignee: Jonathan Hsieh
>            Priority: Blocker
>             Fix For: hbase-6055
>
>         Attachments: hbase-7339.patch, hbase-7339.v2.patch, pre-hbase-7339.patch, pre-hbase-7339.v2.patch
>
>
> Steps:
> - Have a single region table t with 15 hfiles in it.
> - Snapshot it. (was done using online snapshot from HBASE-7321)
> - Clone a snapshot to table t'. 
> - t' has its region do a post-open task that attempts to compact region.  policy does
not compact all files. (default seems to be 10)
> - after compaction we have hfile links and real hfiles mixed in the region
> - t' starts splitting
> - creating split references, opening daughers fails 
> - hfile links are "split", creating hfile link daughter refs.  {{<<hfile>\-<region>\-<table>>.<parentregion>}}
> - these "split" hfile links are interpreted as hfile links with table {{<table>.<parentregion>}}
-> {{<<hfile>\-<region>>\-<<table>.<parentregion>>}}
 (groupings interpreted incorrectly)
> - Since this is after the splitting PONR, this aborts the server.  It then spreads to
the next server.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message