hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
Date Thu, 17 Nov 2016 22:46:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15675035#comment-15675035
] 

stack commented on HBASE-14123:
-------------------------------

More feedback from usage.

So, in backup usage, the backup root dir is called backup_root. In restore, it is called backup_path.
I presume these are meant to refer to the same thing? 

In restore, the id for the backup is the backup_id but in describe when I am to pass a backup
id, it is called backupId. Ditto on delete.  In the dump of the history, the backup id is
called 'ID'.

Suggest you use same name for the arg everywhere. The inconsistency makes the tool appear
untrustworthy.

Are all backup ids prefixed with backup_? Seems superfluous.

Looking at the backup history. What is Progress?  What is 100 w/l a percent on it? (I asked
about this previous I believe).

Want to fix this log message so it doesn't have java object ids in it? 2016-11-17 14:11:14,639
DEBUG [LoadIncrementalHFiles-2] mapreduce.LoadIncrementalHFiles: Going to connect to server
region=x_1_restore,999adfe23d3bda876af50397a462f7d8-18028,1479420656653.15bd3f57fb7364c5d738f8bf7be7a32c.,
hostname=ve0540.halxg.cloudera.com,16020,1479274141369, seqNum=2 for row 999adfe23d3bda876af50397a462f7d8-18028
with hfile group [{[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/33a9cac5e3944e338005dc87e70f85fb},
{[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/935f048447394f93856e917b8dc34a8f}]

I see this in the log:

{code}
  19 2016-11-17 14:13:36,596 INFO  [main] util.RestoreServerUtil: Creating target table 'x_1_restore2'
  20 2016-11-17 14:13:36,596 DEBUG [main] util.RestoreServerUtil: Parsing region dir: hdfs://ve0524.halxg.cloudera.com:8020/user/stack/backup/backup_1479419995738/default/x_1/archive/data/default/x_1/049560f124c122186f174168988f4eec
  21 2016-11-17 14:13:36,598 DEBUG [main] util.RestoreServerUtil: Parsing family dir [hdfs://ve0524.halxg.cloudera.com:8020/user/stack/backup/backup_1479419995738/default/x_1/archive/data/default/x_1/049560f124c122186f174168988f4eec/test_cf
in region [hdfs://ve0524.halxg.cloudera.com:8020/user/stack/backup/backup_1479419995738/default/x_1/archive/data/default/x_1/049560f124c122186f174168988f4eec]
  22 2016-11-17 14:13:36,615 INFO  [main] hfile.CacheConfig: Allocating LruBlockCache size=5.29
GB, blockSize=64 KB
  23 2016-11-17 14:13:36,628 DEBUG [main] hfile.CacheConfig: Trying to use  Internal l2 cache
{code}

Is util supposed to be using a block cache? Seems to be doing this a lot (Lots of spew though
restoring a one row table).

What does this mean?

 253 2016-11-17 14:13:39,782 DEBUG [main] util.RestoreServerUtil: File hdfs://ve0524.halxg.cloudera.com:8020/user/stack/backup/backup_1479419995738/default/x_1/archive/data/default/x_1
on local cluster, back it up before restore

Is this a full copy of the backup to elsewhere?

 296 2016-11-17 14:13:47,907 DEBUG [main] util.RestoreServerUtil: Copied to temporary path
on local cluster: /user/stack/hbase-staging/restore

Why this, the 62minute timeout? Is it excessive?

 298 2016-11-17 14:13:47,955 INFO  [main] util.RestoreServerUtil: Setting configuration for
restore with LoadIncrementalHFile: hbase.rpc.timeout to 62 minutes, to handle the number of
files in backup /user/stack/hbase-staging/restore

Why when I run an incremental backup, I get summary from the FullTableBackupClient? I'd think
I'd get report from the incremental version. I'd also expect a summary of what was in the
incremental?

I'm trying to figure out what these tools are doing when they run from log messages but not
sure. Is there a writeup somewhere? Looks like the technical background in user guide could
be beefed up some. Later.

After reading the user doc., I am unclear on how to restore from a full backup and two incrementals.
I have to manually apply these in order manually? Is there a way to ask the tool to do this
for me?






> HBase Backup/Restore Phase 2
> ----------------------------
>
>                 Key: HBASE-14123
>                 URL: https://issues.apache.org/jira/browse/HBASE-14123
>             Project: HBase
>          Issue Type: Umbrella
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Blocker
>             Fix For: 2.0.0
>
>         Attachments: 14123-master.v14.txt, 14123-master.v15.txt, 14123-master.v16.txt,
14123-master.v17.txt, 14123-master.v18.txt, 14123-master.v19.txt, 14123-master.v2.txt, 14123-master.v20.txt,
14123-master.v21.txt, 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, 14123-master.v28.txt,
14123-master.v29.full.txt, 14123-master.v3.txt, 14123-master.v30.txt, 14123-master.v31.txt,
14123-master.v32.txt, 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, 14123-master.v36.txt,
14123-master.v37.txt, 14123-master.v38.txt, 14123-master.v5.txt, 14123-master.v6.txt, 14123-master.v7.txt,
14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch,
HBASE-14123-v1.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch, HBASE-14123-v12.patch,
HBASE-14123-v13.patch, HBASE-14123-v15.patch, HBASE-14123-v16.patch, HBASE-14123-v2.patch,
HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, HBASE-14123-v7.patch,
HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message